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Introduction 



Many of America’s children are on track for success, but far too 
many are falling behind. While some of our nation’s most precious 
resources — our future doctors, teachers, and engineers — are 
reading above grade level, solving complex algebra equations, and 
applying what they learn in the classroom beyond the schoolhouse 
walls, many are struggling to keep up. Some have simply stopped 
trying.Too many have difficult life circumstances. Others do not see 
the connection between school and their dreams. In America today, 
one in four children fails to graduate from high school on time. 

Even fewer finish college. 

A student’s decision to drop out of high school is not a sudden 
act, but a slow process of disengagement over a period of years. 
With good research in recent years, it is clear that warning signs 
of dropping out are apparent well before students actually leave 
school, signaling the gathering storm of trouble for some as early as 
the elementary or initial middle grades. Research also shows that 
most students at risk of falling offtrack could graduate if they were 
provided with the appropriate supports early enough and those 
supports were sustained. 

Over the past decade, schools, districts, and states have become 
increasingly savvy with data collection and analysis to drive student 
outcomes.The development and use of Early Warning Indicator and 
Intervention Systems (EWS) are at the cutting edge of the data- 
driven, outcomes-focused, high-impact education movement.These 
systems can increase educators’ effectiveness by helping them use 
data to identify those students who are on track to graduate, and 
those who are falling behind, far enough in advance to provide 
appropriate interventions. 



The stakes are high. Dropping out has significant consequences to 
individuals, communities, and the nation. The dropout crisis costs 
individuals millions of dollars in lost income over their lifetimes. 
Communities with high dropout rates also tend to have the 
most unforgiving labor markets, making it difficult to find work 
without the necessary credentials. Dropouts tend to be missing 
from the civic lives of their communities, with much lower rates 
of volunteering, voting, and other indicators of civic health. The 
U.S. economy loses billions of dollars in revenue from a lack of 
productive workers and increased social services. At a time when 
America suffers a serious skills gap and U.S. companies struggle to 
find enough qualified workers, the dropout crisis threatens U.S. 
global competitiveness. 

For too long, educators have lacked important data on key signs 
of early challenges for students. As a result, students with needs 
are sometimes ignored altogether or supports are provided to 
large numbers of students who are not at risk of dropping out, 
or not provided to those who need it most.This misallocates 
educators’ efforts and precious educational dollars. Now, EWS 
can help change that. Leaders in education reform, including those 
championing the Civic Marshall Plan to build a Grad Nation, have 
identified EWS as a key tool to promote student achievement and 
a benchmark against which the nation can chart its early progress 
in stemming the dropout tide. 

This report represents the first national assessment of EWS at 
the district, state, and national levels. 1 It shares evidence from the 
latest research and best practices from the field so that parents, 
educators, administrators, business leaders, and legislators can be 
better equipped to keep children on track to graduate high school, 
prepared for college and career success. 
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Early Warning Indicator and Intervention Systems (EWS) are an 
evolving strategic response to one of our nation’s most pressing 
challenges: enabling all students to stay on track to graduate from 
high school ready for college and career. In an era of data-driven 
education reform, EWS are at the cutting edge. Their rapid 



development is a harbinger of how schools, students, educators, 
and communities can be empowered to use data to help children 
achieve their dreams for generations to come. 

EWS use “real time” or “near real time” data to identify students 
who are off track, so that educators can appropriately support 
them in advancing from grade to grade, and eventually in graduating 
from high school with their class.These systems grew out of 
a simple premise that disengagement from school is a gradual 
process and that students send identifiable signals that they are on 
the path to dropping out. As a result, data could be used to identify 
trends among students, enabling educators to intervene with 
those who are likely to leave the education system unless they are 
effectively supported. 2 From an initial focus on dropout prevention, 
EWS are rapidly evolving toward even broader usage, with 
emergent efforts underway to examine and ultimately integrate 
both school readiness indicators at the start of student’s schooling, 
and college and career readiness indicators during K-12 schooling. 




What is an Early Warning Indicator 
and Intervention System (EWS)? 



Early Warning Indicator and Intervention Systems represent a collaborative approach among educators, administrators, parents, 
and communities to using data effectively to keep students on the pathway to graduation. The best EWS are characterized by a 
combination of features that enable rapid identification of students who are in trouble; rapid interventions that are targeted to 
students’ immediate and longer-term need for support, redirection and greater success; the frequent monitoring of the success of 
interventions; a rapid modification of interventions that are not working; and shared learning from outcomes. 
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Nearly a decade of research supports the predictive power of 
EWS, and education leaders who champion the Civic Marshall 
Plan (CMP) to build a Grad Nation identified EWS early on as 
a key strategy to help drive student achievement.The CMP calls 
for EWS implementation and integration into schools across the 
country — especially the over 1,600 “drop out factory high schools” 
that struggle to graduate a majority of their students on time, the 
3,000 high schools in which 61 to 75 percent of students don't 
graduate on time, and their feeder middle and elementary schools. 3 
(Please see Appendix I orvisitwww.civicenterprises.net/ 
reports for additional information on the Civic Marshall Plan to 
Build a Grad Nation, which has two goals: 90 percent high school 
graduation rate by the Class of 2020 — those starting fourth grade 
this year — and later years, and the highest college attainment rate 
in the world.) 




Though research-based, EWS are an evolving practice still at the 
vanguard of data usage. As a result, a need exists to share leading 
research, to document best practices, and to capture lessons 
learned from the field. That is why Civic Enterprises and the 
Everyone Graduates Center at Johns Hopkins University have 
come together to produce the first national assessment of EWS at 
the local, state, and national levels. Over the last year, we conducted 
site visits and interviews in 16 selected districts and communities 
and in seven states. These visits were informed by a research 
review of hundreds of scholarly articles. 

From Detroit to Dallas, and Massachusetts to California, common 
trends and cautionary tales emerged about EWS design and 



implementation. This report provides an overview of the research 
informed by conversations with teachers working on the front 
lines, district and state officials in the process of building EWS, 
nonprofits working with school systems to implement EWS, 
and leading researchers working to refine and extend early 
warning indicators. We then outline emerging best practices and 
policy recommendations, so that advocates for children from 
the schoolhouse to the White House can apply the best in data 
innovation to their work, with the hope of accelerating high 
school graduation rates, improving college and work readiness, 
and ultimately strengthening American competitiveness in an 
increasingly global economy. 

LESSONS FROMTHE RESEARCH 

The development of predictive early warning indicator data 
systems has occurred rapidly over the past decade. In the early 
2000s, researchers from the Consortium on Chicago School 
Research, the Center for Social Organization of Schools at Johns 
Hopkins University, and the Philadelphia Education Fund set out 
to identify the most highly predictive factors of dropping out. 

They identified three key factors — the ABCs — that were better 
predictors of student outcomes than demographics or test scores: 
Attendance, Behavior, and Course passing/ performance. 45 In the 
last five years, their initial findings have been validated many times 
with state and large district longitudinal studies in Arkansas, Boston, 
Colorado, Florida, Indianapolis, Metropolitan Nashville Public 
Schools, and Tennessee. 6 Research has also identified consistent 
thresholds for these indicators, though they should be assessed 
prior to implementation based both on local goals and policies. 
They indicators and their thresholds are: 

• Attendance: Missing 20 days or being absent 10 percent of 
school days; 

• Behavior: Two or more mild or more serious behavior 
infractions; and 

• Course performance: An inability to read at grade level 
by the end of third grade; failure in English or math in sixth 
through ninth grade; a GPA of less than 2.0; two or more 
failures in ninth grade courses; and failure to earn on-time 
promotion to the tenth grade. 
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LESSONS FROMTHE FIELD 

The EWS field is dynamic. The early adopters of EWS at the state, 
district, and local levels have implemented EWS in a variety of ways. 
With few “one size fits all” prescriptions, in the conversations and 
site visits with 16 selected districts and communities and in seven 
states, common threads tied together the creation, design, and 
implementation of EWS. We found that to be successful: I) vision 
and mission matter; 2) technical components must be strong; and 
3) resources must be assembled and maximized, fully engaging 
stakeholders to use data to guide intervention at the scale and 
intensity required. 

States, Districts, and Schools Collect and 
Use Data in aVariety ofWays 

States vary in how much, and how frequently, data are collected 
and redistributed to users. A third of the states currently have all 
the information for an EWS in their state longitudinal data systems, 
while a third have no plans to collect it. Only a few states explicitly 
consider EWS as a part of the state accountability system. Others 
recommend an opt-in approach. Some states collect district and 
school data weekly and daily, in “near real time” or “real time.” 
Others collect this information once a year. A few states and 
districts redistribute data to the schools in “near real time.” Others 
have yet to do so. Locally, some schools have created systems 
independently, without district or state support or mandate. Some 
EWS are highly technical. Others are done by hand. In sum, data are 



being collected and used differently across the country — and we 
can learn from these lessons. 

• 18 states collect discipline data daily, 17 states collect 
enrollment data daily (which students are supposed to be 
in school), 12 states collect attendance data daily (which 
students are actually in school), and I I states collect 
enrollment, attendance, and discipline data daily. Five states — 
California, Colorado, Illinois, New Jersey, and New York — do 
not collect student-level attendance at all. 

• 16 states produce early warning reports while another 16 
states report that they have no plans or have no set date for 
doing so. 7 Of those producing the reports, only four states 
report that they distribute them to educators on a weekly 
or daily basis. 8 Such early results in the use of state systems 
for early warning reports show some of the challenges of 
aggregating data in a timely fashion at the state level and 
sharing that data with educators and other stakeholders 
who can take appropriate action. 

Vision and Mission Matter 

The site visits showed that Early Warning Systems are, almost by 
definition, promoted in places that have been able to make their 
commitment to educational improvement explicit — and for the 
long haul. EWS reflect a belief that every student can succeed 
with the appropriate support. Barriers also exist to the full and 
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successful implementation of EWS.To be able to effectively use 
attendance, behavior and course-performance data, school-level 
leadership must share a collective vision that graduation rates can 
be improved through the timely and data-driven identification of 
students who require additional supports paired with an organized 
system to provide them. School-based personnel must buy into 
the concept of the expanded data-driven culture. Leadership 
from superintendents and principals matters, with engagement of 
teachers, counselors, parents, and community stakeholders. 

• Louisiana and Virginia have set high standards for vision and 
mission by explicitly considering EWS as part of the state 
accountability systems. 

• Several other states including South Carolina, Tennessee, and 
Texas, have demonstrated their commitment to building EWS 
capabilities and using them to support on-going graduation 
rate and school improvement efforts. 

• Other states, including California, Massachusetts, and 
Washington, have launched pilot or model programs with the 
goal of later expansion. 

Technical Components Must Be Strong 

Data, data usage, and system design can promote student 
achievement. Before an educator can deploy resources effectively 
to the students who need them, he or she must be able to 
accurately identify which students are on track and which are 
off track to graduate with indicators that research tells us are, in 
fact, predictive. The issues of data usage, by whom, and for what 
purposes, while protecting privacy and ensuring compliance with 
existing regulations, can combine to either overwhelm users or 
streamline interventions. 

• Inaccurate data produce incorrect recommendations for 
intervention, citing too many, too few, or misidentified 
students as “off track” and for the wrong reasons. The ad-hoc 
organization of data entry can reduce accuracy. To address 
this challenge, some early warning data systems, such as the 
Metro Nashville system, have built in validation levers with an 
opportunity for data correction. 



• Too much information can reduce buy-in and ownership at 
the school level. Several solutions have emerged to reduce 
the burden of too much information including a focus 

on reporting around a few key research-based predictive 
indicators. In some schools, color-coded “support lists” are 
provided for teams of educators that outline which students 
are struggling to show up, which students misbehave, which 
students are struggling academically, and who is receiving 
which intervention with what result 

Resources Must Be Assembled and 
Maximized 

Data points alone are not sufficient. To turn data into results, 
appropriate interventions must be assembled and human capital 
must be available. Building intervention systems that meet the scale 
and scope of the identified needs are essential to the long-term 
success of EWS. 

• EWS need to inform, and be integrated into, school 
improvement efforts. Ultimately, the goal of EWS is to ensure 
that students have the support needed to succeed in school. 
This will involve both increased and more tailored individual 
supports, and practice and policy improvements at the 
classroom, school, district, and state levels. If EWS are just 
viewed as stand-alone, add-ons for high-needs students, their 
full power and impact will not be realized. 

• Resources to support effective EWS exist, both inside and 
beyond the schoolhouse walls.Teachers can be organized into 
more effective teaching teams; educators can make better use 
of existing support personnel; and community organizations 
can provide wrap-around services like mentoring and tutoring 
or refer students for more intensive supports. Public-private- 
nonprofit partnerships can be key to maximizing existing 
resources and identifying new ones. 

• Stakeholders can share knowledge, needs, and ideas. They 
can serve as mutual resources, from initial design through 
refinement of EWS to meet needs in cost-effective ways. For 
the last five years, educators and EWS developers have been 
constantly learning from each other — across state, district, 
and school boundaries — exchanging information that assists 
each organization in improvement. 
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PATHS FORWARD 

Stakeholders from all sectors, including parents, educators, 
administrators, legislators, business leaders, and funders, can apply 
the lessons learned about EWS to the content of their work. 

To inform this work, we outline emerging best practices, 
highlight areas of needed research, and make policy 
recommendations at the local, state, and national levels. 




Emerging Best Practices: Four Key 
Questions Were Identified, and 
Recommendations Made, for Successful 
EWS Planning and Implementation. 



° Record data from the simplest and most direct 
source possible. 

° Ensure data are entered by appropriately trained 
personnel following well-designed protocols. The 
quality and utility of a data system depends on the 
accuracy of the data stored within the system. Data 
must be consistently coded and coding protocols 
followed daily. 

• Key Question 3: How Will EWS Data Be Used? Once 
data are in a system, they must be converted into 
actionable reports — and these reports should inform the 
work of educators. 

° Use the advantages of technology to compile 
information into easy-to-understand data 
presentations.Transparency and usability should be the 
goals for these reports. 

° Explore issues of privacy. Ensure that children's privacy 
is protected while also leveraging data to effectively 
promote their success. 

° Teach people how to understand and use data and 
provide follow-up coaching for data use. Provide 
training and professional development to help 
educators and administrators learn how to leverage 
the power of data effectively. Compose a “support 
list” of students, revise it every few weeks, and act 
on that data. 



Key Question I : What Types of Data Will Be Recorded? 
When implementing or updating an EWS, stakeholders must 
determine what data can — and should — be recorded. 

° Put the student first. Data helps to identify students 
and craft interventions, but the success of the student 
is the ultimate goal. 

° Use research-based indicators and thresholds and 
respond to student behavior well before triggers for 
more intensive interventions are reached. 

Key Question 2: How Will Data Be Recorded? 
Stakeholders must consider what data will be kept, and by 
whom, when implementing or updating an EWS. 

° EWS can be implemented as early as the later 
elementary school years and should cover key 
transitions (i.e. sixth and ninth grade). 



Key Question 4: Who Should Be Consulted in the Planning 
and Implementation Phases of EWS and What Are Their 
Roles? Stakeholders from the education, private, public, and 
nonprofit communities can do more to promote student 
achievement together than they can alone. 

° Provide local leadership for EWS. Every early warning 
indicator and intervention system needs a champion 
who will advocate for it constantly at the school, 
district or higher level. 

° Have a development and implementation plan 
and timeline. 

° Listen to the end-users and find out what they want 
before going too far. Convene focus groups and build 
up from a pilot. 
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° Integrate EWS into instructional improvement efforts 
and other student support services. High performing 
EWS link efforts to keep students on the graduation 
path with school-wide efforts to improve instruction. 

° Engage stakeholders beyond the school system. EWS 
are only effective when paired with research-based 
interventions. Partnerships matter — from design of 
data systems to collaborations on interventions. 

Needed Research to Enhance the 
National Knowledge Base 

Much is known about early warning indicators, but much less 
is known about the complex set of conditions and policies that 
underlie the most effective implementation and outcomes. Three 
actions are recommended to advance the knowledge in the field: 

* Create federal, state, and district-level workgroups 
to study and guide dropout prevention and 
graduation improvement efforts with a specific 
focus on policies that may contribute to or distract 
from increasing high school and college and 
career readiness rates, on a state- by-state basis. 
Substantial differences exist surrounding the conditions of 
education in the different states.These policies are worthy 
of reexamination in light of the renewed U.S. commitment to 
graduating greater percentages of students who are better 
prepared. 

* Conduct state and district surveys. Determine the 
extent to which early warning indicator and intervention 
systems are being implemented in districts and schools, and 
the characteristics of the districts that have implemented 
these compared with the characteristics of those that 
have not. 

* Conduct design studies, multi-site implementation 
analyses, and random control trials to determine 
the characteristics of the most effective early 
warning indicator and intervention systems. Despite 
the fact that early warning indicator and intervention systems 
are based on research and appeal to common sense, a well- 
controlled study that examines impact has not yet been 
conducted. As part of the scale up of EWS, sound research 
on their effectiveness, under different conditions, should be 
conducted as soon as possible. 



Policy Recommendations to Accelerate 
the Effective Use of EWS for Local, 
District, State, and Federal Stakeholders. 

Policymakers can significantly advance this ground-level work by 
enacting high-impact levers at the district, state, and federal levels. 
Highlights of these recommendations, which are organized around 
the Civic Marshall Plan’s three key recommendation areas, are 
listed here. 

Take Action within Low Graduation Rate Communities. 

* Engage all stakeholders, including nonprofits and 
corporations. Nonprofit organizations have much to offer 
schools as bases for research, as sources of experienced 
planners, and as providers of interventions when schools 
require additional capacity and resources. The business 
community has much to contribute to the successful design 
and implementation of EWS, especially in terms of human 
capital and technical expertise. 

* Link EWS to the ABCs. Research has consistently shown 
that Attendance, Behavior, and Course Performance are the 
strongest predictors of whether a student is on-track to 
graduate. Require the use of data-driven and research-based 
criteria to define each of these three dropout indicators. Tie 
these ABC indicators, and the research-based cut points, to 
EWS and high school completion efforts. 

* Build the infrastructure to support the 
development of EWS at all grade levels. Align 
graduation acceleration efforts with college and career 
readiness efforts through the Common Core Standards 
initiative. Drive attention to the later elementary, middle, and 
ninth grades. 

* Ensure that federal funds, especially Title I dollars, 
are targeted to the highest need schools. This can be 
accomplished by providing additional financial support to schools 
in which 75 percent or less of the senior class graduates. 

° Target funds and attention to districts and 
schools most in need, expanding the focus 
from urban districts to rural districts and 
schools that may not have technical or 
personnel capacity for EWS. Rural schools 
looking for resources can also turn to Rural 
Development State Offices for support as well as 
to The Rural Education Achievement Program (REAP) 
Rural and Low-Income School Grant Program for funds. 
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Build and Enable State and District Capacity to Improve 
Graduation and College Readiness Rates. 

* Invigorate existing policies that encourage 
graduation and implement best practices. Examine 
policies that may act as a disincentive to graduate, including 
double jeopardy policies that react to one negative indicator 
(e.g. poor attendance) by enacting another (e.g. suspension). 

* Ensure that investments in developing the data 
systems needed for EWS are consistently paired 
with efforts to build intervention systems and 
policy reviews, particularly at the state and 
federal levels. Public and private grant makers should 
encourage the integration of early warning indicator and 
intervention systems with related indicator and data-driven 
improvement efforts. 

° Require EWS for federal education funds. 

Require states, districts, and schools with less than 75 
percent cohort graduation rates, and which receive 
federal or state school improvement grants of any 
kind, to implement an early warning indicator and 
intervention system based on longitudinal data analysis. 

° Use funding streams to support efforts on 
the ground, such as Title I Section 1003(g) of ESEA, 
Title I Part A of Elementary and Secondary Education 
Act (ESEA), and Title I Section 613 of Individuals with 
Disabilities Education Act (IDEA). 

° Amend federal regulations involving the 
Supplemental Educational Services section 
ofTitle I to encourage and enable EWS either 
through re-authorization of ESEA or the emerging 
waiver process. 

* Collect and share data and best practices by 
convening a summit of early adopters of EWS, 

including school, district, and state representatives, as well as 
community-based organizations and legislators, to share best 
practices and accelerate effective EWS usage. 

* Improve data collection. 

° Collect information on the status and challenges of 
EWS implementation at the state level through the 
addition of EWS-focused questions in the annual Data 



Quality Campaign survey. Carry out similar efforts with 
districts through the National Center for Education 
Statistics (NCES) or other organizations. 

° Require states to collect information from school 
districts on chronic absenteeism and require school 
districts with high levels of chronic absenteeism to 
develop district-wide plans to reduce it.The re- 
authorization of ESEA Title I, Part A, Section I I I I should 
be amended to ensure that student attendance and 
chronic absenteeism are effectively tracked and those 
students with attendance rates less than 90 percent are 
accurately identified as at risk of dropping out. 

Accelerate Graduation Rates by Strengthening the Public 
Education System. 

* Keep data-based dropout prevention and recovery 
in the forefront of public attention, particularly 
because of the economic downturn. For minimal 

up front costs EWS can help nonprofits, schools, districts, 
and states better target their resources to students at risk 
of dropping out. EWS can minimize the costs of sweeping 
programming and help provide interventions to those 
most in need. 

* Update state and federal accountability systems 
to leverage the highest-impact data. This can be 
done initially through waivers, and permanently through the 
reauthorization of ESEA. Any states that receive relief from 
the current accountability systems should be held accountable 
for having a strong EWS. The reauthorization of ESEA, Title I, 
Part A should then ensure that states, districts, and schools 
have the resources they need to have strong EWS. 

* Clarify and better communicate the implications of 
Family Educational Rights and Privacy Act (FERPA) 
to stakeholders. The protection of children’s privacy must 
be balanced with the need for stakeholders’ support. When 
issuing new rules surrounding FERPA before the end of 201 I, 
the Department of Education should make clear that data 
systems can be tailored to ensure that parents, nonprofits, 
professional providers, and other state and local agencies only 
have access to the data they require to support the students 
they serve. 
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Part I: The Purpose, Current 
Design, and Potential of EWS 



Early Warning Indicator and Intervention Systems (EWS) are an 
evolving strategic response to one of our nation’s most pressing 
problems: the high school dropout crisis and the need to graduate 
high school students who are ready for college and career 
success. More broadly, EWS are also part of a wider evolution 
of an education system that is re-inventing itself, and using data 
along with other tools to move from simply setting standards and 
hoping students succeed to providing clear pathways to success.To 
support students and ensure they have the skills necessary to fully 
participate in the 2 1 st century workforce, educators, parents, and 
communities need tools to rapidly identify and successfully support 
students — as soon as they show signs of being off-track.The last 20 
years have seen an increase in the use of data to drive educational 
improvement in states, districts, schools, classrooms, and at the 
individual student level, from kindergarten through college. Our 
investigation indicates that EWS are among the most promising of 
next generation strategies for using data to increase educational 
outcomes, as they provide educators with the predictive tools 
and response systems they need to guide decisions and support 
student success. The value of using EWS is just beginning to be 
recognized as a key to how schools, students, educators, and 
communities can use data to help children achieve their dreams. 



THE NEED -THE DROPOUT CRISIS IN 
HIGH SCHOOLAND COLLEGE 

While the nation has made progress in boosting high school 
completion rates in some states and communities, the rate of 
progress is too slow to meet national goals. 9 With a national 
graduation rate of 75 percent in 2008 (up from 72 percent in 
200 1), 10 one-fourth of all public high school students and 40 
percent of minorities still fail to graduate on time." Further, the 
dropout rate exceeds 50 percent in many of the nation's most 
challenged urban and rural schools. Nationally, the high school 
dropout crisis claims more than one million students each year. 12 
Of the students who do complete high school, only 70 percent 
enroll in a postsecondary institution. 13 14 Of this 70 percent, only 
57 percent complete a bachelor’s degree within six years, 15 and 
only 28 percent complete an associate’s degree within three 
years. 16 In some high poverty urban schools, the college graduation 
rate is often in the single digits. 17 
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> — “ The dropout rate exceeds SO percent in many of the nation’s 
most challenged urban and rural schools. The cost of our 

nation’s education achievement gap is equivalent to two to 

four percent of GDP and imposes the “ economic equivalent of 
a permanent national recession.” 18 



The stakes for improving the U.S. educational system and 
reducing the dropout rate are high. A labor market skills gap 
accompanies the crisis in high school and college completion. 

The majority of job openings in the next decade will require at 
least some postsecondary education. 19 Experts estimate that 
American businesses are in need of 97 million middle- and 
high-skill workers, yet only 45 million Americans possess the 
necessary education and skills to qualify for these positions. 20 
The lagging rates of high school completion are linked to broader 
societal and economic impacts including lost wages, taxes, and 
productivity, along with the financial stability of individuals and 
their quality of life, the economic health of our rural areas, towns, 
cities, states, nation, and U.S. global competitiveness. A high school 
dropout earns, on average, about $260,000 less than a high school 
graduate over a lifetime. 21 Lost wages and accompanying social 
costs accumulate. In fact, a 2009 report estimates that the cost of 
our nation’s education achievement gap is equivalent to two to 
four percent of GDP and imposes the “economic equivalent of a 
permanent national recession.” 22 

THE R ESPONSETO THE NEED - THE 
CIVIC MARSHALL PLAN TO BUILD A 
GRAD NATION 

Accelerating high school and college completion rates will help 
to secure the future of individual Americans, households, and the 
nation. Beginning in 2010, education, policy and community leaders, 
including the America’s Promise Alliance, Civic Enterprises, the 
Everyone Graduates Center at Johns Hopkins University, and 
the Alliance for Excellent Education came together around Grad 
Nation, a 10-year campaign to end the high school dropout crisis. 23 
The engine of the Grad Nation initiative is the Civic Marshall 
Plan (CMP), 24 a plan of action 
to meet the Grad Nation goal 
of a 90 percent high school 
graduation rate nationwide for 
the Class of 2020 (those in fourth 
grade during the 20 1 I -20 1 2 
school year) and beyond, with all 
graduates ready for college and 
the 2 1 st century workforce. 



Raising the national graduation 
rate 1 5 percentage points in 
nine years, and by 30 percentage 
points in states with current 
graduation rates hovering at 
60 percent, is a formidable, but 
not insurmountable challenge. 
Statistics and case studies presented in Grad Nation: Progress and 
Challenges (20I0) 26 and Grad Nation: Progress and Challenges, Annual 
Report 201 1 , 27 demonstrate that sustained increases in graduation 
rates of three percentage points per year have occurred in several 
states and districts over the last six years.These successes resulted 
from clear goals, high expectations, smart education policies, good 
use of data, supports for educators and students, and multi-sector 
collaboration. Continued success will be possible as the lessons 
learned, which are documented in these publications, are adopted 
and refined. 

The Civic Marshall Plan and Early 
Warning Indicator and Intervention 
Systems 

The Civic Marshall Plan (CMP) focuses on using the main, evidence- 
based levers for addressing the dropout problem, improving the 
graduation rate, and increasing college attainment. It emphasizes a 
multi-sector approach that engages national, state, and community 
stakeholders in different roles to affect individual, community, 
state, and national outcomes. Spearheaded by four convening 
partners, the effort is also guided by a CMP Leadership Council 
that grew out of 20 1 0 roundtable discussions among three key 
constituencies: I) policymaking institutions, such as the National 
Governors Association and the National Conference of State 
Legislatures; 2) associations representing educators, such as the 
National Association of Secondary School Principals, the American 
Association of School Administrators, the American Federation 
ofTeachers, and the National Education Association; and 
3) community-based organizations, including City Year, 

Communities In Schools, United Way, Big Brothers and Big Sisters, 
and the Boys & Girls Clubs, as well as key national foundations and 



> 



The engine of the Grad Nation initiative is the 
Civic Marshall Plan (CMP), 15 a plan of action to meet the 
Grad Nation goal of a 90 percent high school graduation rate 
nationwide for the Class of 2020 (those in fourth grade during 
the 201 1-2012 school year) and beyond, with all graduates 
ready for college and the 2 1st century workforce.The CMP 
identified EWS early on as a key tool to accelerate student 
achievement and a benchmark against which the nation can 
chart its early progress in stemming the dropout tide. 
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corporations. This group is a highly-focused, targeted collaboration 
committed to transforming the lowest-performing schools and 
providing pathways to postsecondary success for students. They 
are aligning mission-focused efforts with the CMP framework 
and benchmarks, executing these efforts, and supporting strong, 
cohesive national and state policies that advance the CMP.The 
CMP framework includes 1 6 research-based strategies 28 for 
improving the graduation rate by 2020, coupled with indicators and 
annual reports for tracking and reporting progress. 29 EWS were 
identified early in this campaign as a key strategy to help drive 



student achievement.The CMP calls for EWS implementation in 
the nearly 1,700 high schools with 60 percent or lower graduation 
rates (“dropout factories,” found in 49 states) and their feeder middle 
schools by the beginning of the 2012-2013 academic year, 30 31 and the 
more than 3,000 high schools with marginally better graduation 
rates (61 to 75 percent) that are still below the national average. 
(For additional information on the Civic Marshall Plan to Build a 
Grad Nation, please see Appendix I and visit 
www.civicenterprises.net/reports.php.) 



A Note to Educators 



1 

A 



Children’s lives change in classrooms and America’s educators are preparing the next generation of our nation’s leaders. Teachers 
are under increasing pressure to produce results — and prove their effectiveness. We believe, based on conversations with educators 
across the country, that well-implemented EWS can make educators’ jobs more manageable and more influential. EWS are not meant 
to be another reporting mechanism or complex data protocol. Rather, EWS are meant to be an easy-to-use tool to make teaching, 
counseling, and coaching more effective by providing data that enables educators to pinpoint a student’s specific challenges and align 
resources to address them. 



WHAT IS AN EARLY WARNING 
INDICATOR AND INTERVENTION 
SYSTEM (EWS)? 

Early Warning Indicator and Intervention Systems represent a 
collaborative approach among educators, administrators, parents, 
and communities to using data effectively to keep students on 
the pathway to graduation. The best EWS are characterized by a 
combination of features that enable rapid identification of students 
who are in trouble; rapid interventions that are targeted to 
students’ immediate and longer-term need for support, redirection 
and greater success; the frequent monitoring of the success of 
interventions; a rapid modification of interventions that are not 
working; and shared learning from outcomes. These systems 
are implemented at the school level, sometimes in conjunction 
with community partners, and are sometimes supported with 
data supports and analysis from districts and/or states. In 
some emerging cases, these systems are also being used at the 
district level to organize and allocate resources and monitor the 
effectiveness of interventions. 



EWS Premise, Promise, and 
Collective Approach 

EWS grew out of a simple premise that data could be used to 
identify trends among students, enabling educators to intervene 
with those who are likely to leave the education system unless 
they are effectively supported. For generations, highly competent 
teachers, counselors, and administrators have understood and 
acted on the construct that early intervention matters. They 
designed interventions for individual students by evaluating 
information from day-to-day status reports in grade books, 
attendance, and other school records, coupled with anecdotal 
information gained in the hallways. EWS seek to bring this approach 
to scale and build it around evidence-based practices. EWS refine, 
systematize, and redirect data to educators, identifying students 
who would be likely to either dropout or graduate unprepared to 
succeed in college and career. Even more importantly, they help 
guide educators toward the most efficient and effective responses, 
which include not only student-level interventions but also 
classroom, school, and even district and statewide actions. EWS are 
premised on a body of well-documented research that a student’s 
decision to drop out is not a sudden act, but a slow process of 
disengagement over a period of years, with clear warning signs 
apparent well before students actually leave school. 32 
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Definitions 



A 



Early warning systems are an emerging practice that lack standard definitions. The following definitions, which we use in the report, 
are based on our analysis of current usage. Our goal is to generate discussions that lead to standard definitions. 

Off-Track Indicators are measures of student behavior and/or performance linked to empirically derived thresholds, below which 
students have strong probabilities of not achieving essential educational outcomes like graduation from high school — unless effective 
interventions are applied. At a minimum, students below the identified threshold should have a less than a 50 percent probability 
of achieving the desired outcome and some argue for setting a higher bar at 66 percent or even 75 percent to produce greater 
efficiencies. Off-track indicators should also be based on readily available data that is typically measured continuously over time to 
enable tracking of both improvement and further decline. Off-track indicators should have a statistically significant independent effect 
on predicting the desired educational outcome. In the most robust efforts, potential indicators are tested against each other and 
examined in concert to establish the most parsimonious set of indicators with the highest yield (e.g. total number of all dropouts 
predicted). Indicators that are highly predictive, but only identify a small percent of students who fail to achieve the desired outcomes, 
should not be used. Students with off-track indicators are in need of interventions. 

On-Track Indicators are measures of student behavior and/or performance linked to empirically derived thresholds above which 
students have strong possibilities of achieving essential educational outcomes like enrolling in college. They have the same other 
properties as off-track indicators, except it is the absence of the indicator which suggests that interventions are required. Students 
with on-track indicators are not in need of focused interventions like off-track students; but their progress should still be monitored 
to enable rapid response if they begin to slip, as well as to track efforts to accelerate their success. 

Early Warning Flags identify students who are moving towards thresholds of behavior or performance that indicate they are off 
track, or away from thresholds that indicate they are on track.These can be empirically derived but are also sometimes based on 
educator knowledge and common sense. For example, students who have missed 1 0 percent or more of school over several months, 
or students who have failed two major tests, and/or have sudden downward shifts in attendance, behavior, or course performance 
should be flagged. Early warning flags are designed to say: “take notice, monitor, investigate, talk with the student, problem solve, 
and provide opportunities for rapid recovery.” Flags can prevent students from ever achieving an off-track indicator, as well as help 
students stay on track. 

Early Warning Systems (EWS) are organized efforts at the school, district, or state level to provide data systems that can 
identify students with off- and/or on-track indicators combined with systematic and tailored intervention. This data can be shared 
with stakeholders including administrators, teachers, guidance counselors, coaches, nonprofit partners, parents, and students. 
Increasingly, early warning system flags are combined with a systematic approach to diagnosis, enhancing educators’ ability to provide 
the appropriate interventions or supports to the identified students. The highest standard EWS also organize district-, school-, and/or 
classroom-wide prevention activities to keep students from falling offtrack; monitor and track the effectiveness of interventions; link 
use of on- and off-track indicators to efforts to improve classroom instruction, schools and school systems; and analyze and reform 
policies and practices which explicitly or implicitly work against the desired outcome. 

Data-Driven School Improvement includes efforts to use data and data analysis to improve student outcomes. EWS are 
one example of data-driven school improvement, as is the practice of using benchmark assessments. Both strategies benefit from 
discussions by educators organized into professional learning communities, enabling improved classroom practice and developing 
more efficient and effective means to support students by allocating resources at the classroom, school, district, and state level. 
Coaches — master educators with time allocated to guide their colleagues in the use and interpretation of and reflection on data — 
have proven to be important assets in making data driven school improvement work. 
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> — “ Over a decade of research supports the development of EWS. 
Key early warning indicators and their thresholds are: 

• Attendance: Missing 20 days or being absent 10 percent of 
school days; 

• Behavior:Two or more mild or more serious behavior 
infractions; and 

• Course performance: An inability to read at grade level by 
the end of third grade; failure in English or math in sixth 
through ninth grade; a GPA of less than 2.0; two or more 
failures in ninth grade courses; and failure to earn on-time 
promotion to the tenth grade. 



THE ABCS - PRIMARY, PREDICTIVE 
INDICATORS ARE ATTHE HEART 
OF EWS 

Teachers have long remarked, “I can teach them if they will 
only come to school.” Others have amended that statement to 
include “on time.” Still others add, “if they pay attention and stay 
out of trouble.” Research corroborates these anecdotes and 
demonstrates that there are certain highly predictive indicators. 33 
These are the ABCs of EWS: 

• Attendance, 

• Behavior, and 

• Course performance . 34 

Why the ABCs? Detailed Lessons from 
the Research Base 

The development of predictive early warning indicator data 
systems has occurred rapidly over the past decade. Prior attempts 
to identify potential dropouts and intervene effectively often relied 
on data from a fixed point in time, including a student’s social 
background, educational experiences, and school characteristics. 

In the early 2000s, researchers set out to investigate accurate 
predictors of dropouts. The Consortium on Chicago School 
Research and the Center for Social Organization of Schools at 
Johns Hopkins University, working with the Philadelphia Education 
Fund, assembled retrospective data on multiple longitudinal cohorts 
that enabled researchers to follow 
students over time and study how a 
student’s behavior at one point in time 
impacted later outcomes. The Chicago 
research followed students beginning 
in the eighth and ninth grade forward; 

Philadelphia’s from the sixth grade. Both 



demonstrated that indicators of 
student behavior — the extent to 
which students attended school 
regularly and succeeded in their 
courses — could accurately 
identify the vast majority of 
students who did not graduate. 35 
36 The Philadelphia research 
also showed that sustained 
mild misbehavior, which earned 
middle grade students poor 
behavior marks, also had strong 
predictive power.These analyses 
found key factors — the ABCs — 
were much more predictive 
of student outcomes than demographics or test scores. By 
extrapolation, the data led to identification of thresholds and flags 
that could predict warning signals for future students. 

In addition to identifying the predictive power of the ABCs, the 
Chicago and Philadelphia research showed the importance of 
students having successful transition years in sixth and ninth grades. 
These two major research investigations were then followed by 
analysis from other leading research centers, states, and school 
districts, which independently confirmed the predictive power and 
importance of the ABCs. This work also extended and refined the 
understanding of how early warning indicator systems operate 
across the grade spans. Retrospective studies helped to identify the 
thresholds that predicted whether students were on track or off 
track for high school graduation. Projective studies, which followed 
students out of high school and into the future, linked high school 
performance with post-secondary outcomes. 37 

The emerging work regarding on- and off- track indicators shows 
that in order for all students to graduate college and career ready, 
several key transitions must be navigated and students who struggle 
with them are thrown off the graduation path. In the elementary 
grades, it is critical for students to master key academic skills that 
provide the foundation for future learning.The emerging evidence 
indicates that chronic absenteeism in the early grades inhibits this, 
and hence consistently attending school from kindergarten forward 



> Warning signs of dropping out are apparent well before 
students actually leave school, signaling the gathering 
I storm trouble for some as early as the elementary or 
initial middle grades. 
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matters. The evidence is also clear that grade-level reading by at 
least the end of third grade matters. Further, by early adolescence 
it is critical that students firmly believe that doing well in school 
is important and that they come to school regularly, and engage 
positively in their classes. The existing evidence indicates that from 
the fourth grade on these behaviors begin to shape graduation 
outcomes. In high school, skills and behaviors continue to matter 
but credit accrual is the currency of advancement, and passing 
courses becomes paramount. Our understanding of what it takes 
to successfully make the transition to college and career training is 
just emerging, but existing evidence suggests that good grades and 
strong attendance signal the possession of key attributes. 

Early Grades Indicators 

The predictive power of the ABCs holds true for the early years. 

A longitudinal study of more than 20,000 children in kindergarten 
through the fifth grade in 900 schools across the country found 
high rates of chronic absenteeism and a strong link between 
early absenteeism and later struggles in school. 38 Chronically 
absent kindergartners living in poverty were at the lowest level 
of achievement among their peers by fifth grade. 39 A study of San 
Diego Unified School District fourth-graders followed students 
through their performance seven years later on the eleventh 
grade California High School Exit Exam (CAHSEE), required for 
graduation since 2006. 40 For this group of students, academic 
grade point average (GPA) and classroom behavior (beginning 
work promptly and following directions in the early years, as noted 
in teachers’ records) were the strongest predictors of eventual 
success or failure on the CAHSEE. 

Another longitudinal study of nearly 4,000 students found that 
those who could not read proficiently by third grade were four 
times more likely to leave school without graduating than were 
proficient readers. 41 For the least proficient readers, those unable 
to master basic skills by third grade, the rate was almost six times 
greater. 42 The evidence is clear that reading on grade level matters. 
This indicator alone, however, does not always capture the majority 
of future dropouts. In other words, some students who read well 
in the fourth grade still drop out, highlighting the need to build 
indicator systems that span grade and school levels. 

Middle Grades Indicators 

Longitudinal studies by the Baltimore Educational Research 
Consortium (BERC) confirmed the Philadelphia findings that large 
percentages of future dropouts could be identified as early as the 
sixth grade. 43 The research showed that sixth-graders with one of 
four high-yield early warning indicators had half the graduation rate 



of those who did not. 44 These indicators were chronic absenteeism 
of 20 days or more; failing English and/or math; or a failing average 
for English, math, social studies and science combined. A Boston 
study found that extremely low test scores in the middle grades 
were, together with the ABCs, predictive of high school dropout 
in a state that requires students to pass high school exit exams 
in order to graduate. 45 The Philadelphia and Boston studies also 
showed that in the sixth grade, students who eventually dropped 
out typically failed math or English, not both subjects, and that 
course failure often, but not always, had an attendance and behavior 
covariate. Finally, there was some evidence in the Boston study that 
the earlier a student developed an off-track indicator in the middle 
grades, the greater his or her odds of not graduating. 

High School Indicators 

Studies by the Consortium on Chicago School Research 
determined that ninth-graders who earn five full-year course 
credits and receive no more than one semester grade of F are 
three and a halftimes more likely to graduate from high school 
than their peers who do not.They also found a strong link between 
attendance and course passing. 46 BERC researchers found similar 
outcomes in Baltimore, as did studies across multiple school 
districts in Colorado. 47 Chicago and Philadelphia studies found 
significant percentages of ninth-graders who had not shown 
risk factors in middle grades, yet slid offtrack in ninth grade, 
demonstrating the key significance of transition grades and that 
indicators are needed along the educational continuum. 48 

Validity of the ABCs 

The initial studies described above identified the ABCs as 
the primary predictive indicators. In the last five years, these 
findings have been validated many times, including state and 
large district longitudinal studies in Arkansas, Colorado, 
Florida, Indianapolis, Metropolitan Nashville Public Schools, 
Tennessee, Texas, and Virginia. 49 

EWS Thresholds for the ABCs 

Accurate thresholds are required to successfully use the ABC 
indicators to create actionable reports. The predictive thresholds 
or “cut points” for the ABCs at which students are unlikely to 
graduate without interventions can vary by locale. For instance, 
some studies find that 80 percent attendance is the cut point 
(which in a 180-day school year means that students are missing 
36 days of school), and others 90 percent.This variance occurs 
as a trade-off between yield (identifying a large percent of future 
dropouts) and accuracy (high odds that the students identified will 
not graduate without effective intervention). In different studies, 
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different choices between yield and accuracy have been made. 
Cut points may also vary based on desired outcome: high school 
graduation versus college readiness and enrollment. 



Despite these variations, research-based thresholds for the ABCs 
have been identified, and can be used as a starting point when 
creating an EWS. In summary, they are: 



Local factors and policies also play a role in determining the cut 
points. A few examples of these often confounding factors are 
below, and suggest future research studies. 

• At the state level, are policies reasonable and are they 
enforced equitably or arbitrarily? For example, studies in 
both Massachusetts 50 and Texas 51 show large classroom-by- 
classroom, school-by-school, and district-by-district variations 
in application of state-legislated discipline policies for non- 
violent behaviors, with consequences for graduation. 



• Attendance: Missing 20 days or being absent 10 percent of 
school days; 

• Behavior: Two or more mild or more serious behavior 
infractions; 

• Course performance: An inability to read at grade level 
by the end of third grade; failure in English or math in sixth 
through ninth grade; a GPA of less than 2.0; two or more 
failures in ninth grade courses; and failure to earn on-time 
promotion to the tenth grade. 



• At the district and school levels, do behavior policies and 
attendance policies interact? For example, if a student is 
suspended are they also counted as absent? Are students 
suspended for being truant? Does attendance factor into 
course passing? If students are absent, are they allowed to 
make up work? If they miss a certain number of days do they 
fail the course? 

• How do local, non-educational policies affect students? For 
example, at the district level, do all students ride metropolitan 
transit or do they get on the school bus, making it much more 
difficult to skip school? 

• At the district or school level, what is a failing grade? Is it a 
D or an F, a 55 or a 69? To ensure the highest impact, local 
longitudinal data and stakeholder input concerning local 
factors should be considered, possibly modifying cut points. 



Educators should use knowledge of cut points to trigger significant 
interventions much earlier, before the cut points are reached. In the 
best emerging EWS and most highly effective schools, diagnostic 
and preventive actions are instigated at a few days of absence, one 
behavior infraction, or one slide in grades. 52 

Beyond the ABCs - Secondary and 
Tertiary Indicators 

While the ABCs have been repeatedly shown to be the most 
important indicators, some EWS include other factors. These 
secondary and tertiary indicators are often based on state and 
local expectations. State assessment scores, whether for NCLB or 
state accountability systems, are typically less predictive, but are 
often included and can provide important contextual information 
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Test Scores and Grades as Indicators 



r 



Grades, in particular course failures, are better predictors of a student’s likelihood of graduating on time than achievement test 
scores. There are several reasons for this. For example, achievement tests are given at one point in time, and for a single individual 
can be influenced by a wide assortment of non-academic factors on a given day. Course grades, on the other hand, represent a 
summative judgment of a semester’s or year's worth of work, capturing student performance, effort, motivation, and attendance over 
time. At the high school level, course passing is directly tied to credit attainment, which in turn is linked to grade promotion and 
high school graduation. 

Grades, moreover, are continuously available, and hence can be used to monitor progress or decline, whereas tests are often 
given only once a year, and even less frequently in high school. Some have argued that test scores should have greater weight than 
grades because test scores are less arbitrary than grades, even if they are weaker predictors. This concern, however, at some level 
is comparing apples to oranges. Both are important. Test scores, are important for accountability and to see if students and schools 
s are achieving at desired standards. Grades, however, can help tell us if a student is on track to graduate high school, prepared for 
college and career success. 

The Common Core State Standards provide a means to address some of these concerns. They provide a “consistent, clear 
understanding of what students are expected to learn, so teachers and parents know what they need to do to help them,” as such, 
they can facilitate standards based grading, and the use of common grading rubrics. 55 



to guide interventions. 53 The addition of these indicators can 
lead to the identification of students who have strong academic 
skills but are still failing courses, often because of attendance or 
behavior variables. Overage for grade by 2 1 months or more 
is often included, identifying a sub-set of students who need 
acceleration opportunities to rejoin their peers. 54 Life situations 
such as homelessness, foster care, pregnancy, involvement with the 
juvenile justice system, and the necessity of taking care of a relative, 
are considered by many to be important tertiary indicators. These 
situations are often captured indirectly by attendance data, but 
data that indicate students are involved with agencies (including 
social, judicial, and medical) can provide important context to 
guide interventions, as well as enable inter-agency collaboration in 
support of the students.The trade-off is that the inclusion of social 
service agency data in an EWS can raise questions of confidentiality. 



THE RELATIONSHIP OF EWS TO STATE, 
DISTRICT, AND SCHOOL-LEVEL DATA 
SYSTEMS AND COLLECTION 

EWS are fundamentally a school-based approach to improving 
student achievement. In essence, these systems come down 
to a seemingly simple action: making a list of the students who 
are predicted to need support, and acting on the list. All of the 
fundamental data needed for an EWS originates at the school-level, 
from the attendance taken, the disciplinary forms written, and the 
grades given. It is typically, however, not organized or analyzed to 
identify students who are falling off track. Technology can be critical 
to a more effective system, but in some cases they need not be 
technology-dependent. One Louisiana educator remarked, "when 



> EWS are fundamentally a school-based approach. All of the 
data needed for an EWS originates at the school-level, from 

the attendance taken, the disciplinary forms written, and the 

grades given. 
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The Data Quality Campaign, a national nonprofit that 
supports state policymakers’ efforts to use data to improve 
student achievement, has captured the progress of statewide 
longitudinal data systems. 



I go into a small school, and the principal has a scratch pad with 
names of the students who are getting into trouble, well then, he’s 
doing his job. And when I go into a big school, say 1 ,000 or 2,000 
students, and the principal has one of our bi-weekly state early 
warning system reports on her desk, flagging the ones who need 
help, well then, she’s doing her job.” 56 

Increasingly sophisticated state and district data systems have 
evolved over the last 25 years, however, and their advancement 
has accelerated recently through an infusion of federal funding. 
These systems offer opportunities to consolidate and integrate rich 
data from multiple sources with great potential for action driving 
early warning indicator data analyses and reports. At the same 
time, increasing complexity requires compatibility of hardware and 
software at many levels. This brings with it its own set of challenges. 

State Longitudinal Data Systems (SLDS) 

Since 2005, states have made significant progress building and using 
statewide longitudinal data systems. The Data Quality Campaign, 
a national nonprofit that supports state policymakers’ efforts 
to use data to improve student achievement, has captured this 
progress. According to DQC’s annual survey of states, every state 
now has in place the basic infrastructure of a statewide longitudinal 
data system. 

While the Education Sciences Reform Act of 2002 prohibited the 
U. S. Department of Education from creating a national student 
unit record system, federal policy has played a significant role in 
advancing state and local efforts. That same law authorized the U.S. 
Department of Education to determine voluntary data standards 
and established a federal grant program to assist states in designing 
and using their SLDS. Forty-one states and the District of Columbia 
have received such grants since 2005 and are in varying stages of 
development and enactment.57 
Moreover, in exchange for 
federal stimulus dollars in 
2009, every governor and chief 



state school officer committed 
to having a SLDS in place by 
September 30, 2011. 

From the EWS perspective, the 
virtue of SLDS is that because 
SLDS are centered on unique 
student identifiers, accurate longitudinal research studies can 
establish the characteristics and trajectories of potential dropouts. 
Similarly, profiles of high school graduates can be established and 
their outcomes followed to extend EWS to college readiness. As 
SLDS begin to encompass early childhood, as well as post-high 
school information, there is the potential to construct a systematic 
approach to providing the supports needed for all students to be 
college and career ready. 

State longitudinal data systems, however, are only beginning to be 
used to support early warning efforts. Only a minority of states has 
all the information required by EWS in their state longitudinal data 
systems and these states vary in how much, and how frequently, 
data are collected and redistributed to users. According to DQC’s 
2010 survey, only 18 states collect discipline data daily, 17 states 
collect enrollment data daily (which students are supposed to be in 
school), 12 states collect attendance data daily (which students are 
actually in school) and I I states collect enrollment, attendance, and 
discipline data daily. Five states, California, Colorado, Illinois, New 
Jersey, and New York, do not collect student level attendance at all. 
Updated information from DQC’s 201 I survey will be released in 
late 2011. 

Sixteen states responded that they use this information to produce 
early warning reports while another 16 states report that they 
have no plans or have no set date for doing so. 59 Of those currently 
producing the reports, only four states report that they distribute 
them to educators on a weekly or daily basis. 60 Such early results 
in the use of state systems for early warning reports shows some 
of the challenges both of aggregating data in a timely fashion at 
the state level and of sharing that data with educators and other 
stakeholders who can take appropriate action. 



Sixteen states responded that they use this information to 

produce early warning reports while another 1 6 states report 
| that they have no plans or have no set date for doing so . 58 
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The integration of data from student information systems 
with that from other information systems is among the chief 
challenges in successful EWS implementation. 



Data Confusion 

Education technology capacities 
have expanded dramatically in 
the last 20 years. States, districts, 
and schools have frequently had 
to make hardware and software plans, decisions, and purchases 
without being able to foresee the possibilities that technology 
innovations would bring.They often do so without the funds 
to adapt nimbly, as businesses might. Currently, varying by state, 
district, and school, the location of data and the ease of integration 
and feedback are highly variable. Some states and large districts 
are making advances in creating data warehouses, which store 
data from many different sources over time, and which with the 
appropriate programming can integrate data in response to queries. 
Generally, compliance, assessment, and regulatory reporting, 
tracking, and feedback are largely carried out at the state and 
district level, yet most ABC data needed for an EWS is generated 
at the school-level. 




On the bright side, technology transfer from the business sector 
has facilitated the development and adoption of better education 
technology systems. Lessons learned from the business sector 
enable better capture data from multiple “silos” maintained in 
different state, district and school divisions, by different people, for 
different purposes, with different collection and report-generating 
schedules, with initially incompatible software and sometimes 
different hardware and operating systems. In some cases, users, 
designers, and vendors are increasingly collaborating to generate 
solutions, test those solutions, evaluate, and redevelop the systems. 
In others, a concentrated effort is still required to get data 
originally collected by schools back to educators into an actionable, 
EWS-supported format. 

Some states are moving to reduce the challenge. In Alabama, 
the student information system (SIS) is state-provided, and the 
vendor recently incorporated an EWS at no charge to sites. Texas 
districts have in the past used over 60 SIS, but in a move toward 
simplifying the interface between local SIS and an improved state 
data system, the state in August of 20 1 I identified as preferred 
vendors two vendors that already served approximately half of 
its districts, and offered to support these SIS without cost to 
sites. In Washington state and Wisconsin, members of district 
collaboratives have worked to organize their SIS within the context 
of state requirements. In other states, for instance North and 
South Carolina, SIS are state-designed, mitigating the challenge of 
interfacing district systems. 
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Good Data In. Good Data Out. 

Beyond the technological challenges, for EWS to be a useful tool 
requires that data based on common definitions be accurately 
recorded. There is wide variety across schools, districts, and states 
in how terms like “absent,” "tardy,” and “enrolled” are defined. 
Efforts are underway to help standardize basic definitions. For 
example, the Common Education Data Standards (CEDS) Initiative, 
sponsored by the National Center for Education Statistics (NCES) 
and a voluntary group of stakeholders from multiple states, seeks 
to define a common vocabulary of standard definitions, code 
sets, business rules, and technical specifications that increase data 
operability and portability. A second version of the voluntary 
standards is scheduled to be released in early 2012. 

THEORY OF ACTION: HOW CAN EWS 
IMPROVE GRADUATION RATES? 

EWS represent a systematic use of data that can provide educators 
with tools to significantly accelerate graduation rates. The prospect 
of predicting off-track students as early as the late elementary 
and early middle grades would provide schools with significant 
advantages in identifying those students needing the most help and 
supporting them early and intensively enough to avoid paths that 
lead to dropping out. Reducing negative student transitions at the 



elementary to middle school and middle school to high school 
levels can increase high school completion. These early prediction 
models would allow schools to support students well before they 
enter high school — dramatically improving the likelihood that 
students enter high school prepared for success. Such efforts 
preserve the gains made from investments in early childhood 
education, as well as lay a foundation for success after graduation. 

Identifying off-track students, however, is only half of the purpose 
of these systems. Students must then receive appropriate support 
to ensure they remain on-track. 61 62 63 Schools using these systems 
have been able to more effectively identify students who will 
benefit from services, and avoid focusing dropout prevention 
resources on students who have previously been misidentified as 
at-risk, often because of socioeconomic or demographic factors. 
Such efforts ensure that limited federal, state, and local resources 
are used efficiently and maximize returns on investment. EWS 
allow schools to monitor the effectiveness of interventions and 
continually modify the supports provided until students are back 
on track to high school graduation. Dropout trends identified by 
data systems can help schools and districts to identify systemic 
weaknesses, policies, or practices that may be increasing the 
likelihood of student dropouts, while also enabling schools and 
districts to more effectively target their limited dollars. 64 



> Identifying off-track and on-track students, however, is only 
half of the purpose of EWS. Students must then receive 
appropriate support to ensure they remain on-track. 
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WHAT IS THE CURRENT STATE 
OF EWS? 

Over the last year, representatives of Civic Enterprises and 
the Everyone Graduates Center at Johns Hopkins University 
conducted site visits, interviews, and conversations with 
educators and/or community representatives in seven states: 
Alabama, California, Louisiana, Massachusetts, South Carolina, 
Texas, and Virginia and 1 6 communities: Boston, Chattanooga, 
Chicago, Columbia, Dallas, Detroit, Houston, Knoxville, Memphis, 
Minneapolis, Nashville, New Orleans, Philadelphia, San Antonio, 
Seattle, and St. Louis. 



The team heard from teachers, administrators, district, and state 
leaders in research, assessment and information technology 
divisions, nonprofit and corporate partners, and policymakers to 
determine what trends are evident, what barriers to effectiveness 
exist, and what opportunities presented themselves for new 
developments. From this work, lessons were learned, challenges 
were identified, and recommendations were noted. Other schools, 
districts, and states can learn from these lessons as they work to 
develop or accelerate implementation and use of such systems to 
boost student success and high school graduation rates. 

Site visits and interviews with 
educators and other stakeholders 
across the country made it 
clear that EWS are in a phase of 
dynamic growth and on an upward 
trajectory. Outreach efforts by 
the Chicago Consortium for 
School Research, the Everyone 
Graduates Center at Johns 
Hopkins University, Jobs for the 



> 



This report represents the first national assessment of EWS 
at the district , state, and national levels, including site visits 
and interviews in 1 6 selected districts and communities 
in seven states. To be successful, we found that vision and 
mission matter; technical components must be strong; and 
resources must be assembled and maximized, fully engaging 
stakeholders to use data to guide interventions at the scale 
and intensity required. 
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Future, the National Governors Association Best Practice Office, 
the National High School Center, and others, helped to share initial 
research findings among a diverse set of stakeholders. This in turn 
enabled a wave of early adopters of EWS across the nation, at the 
state, district, school, and community level. These collaborations in 
turn are advancing EWS and demonstrate their potential power to 
improve educational outcomes. 

In addition to state and district efforts, we noted that nonprofits, 
community groups, and corporations are the champions of EWS, 
working in collaboration with schools and districts. We also saw 
that across the country, the education landscape has been in a 
constant state of change: states are adopting new graduation 
rate calculations on different timelines in order to comply with 
the uniform federal calculations required for 2010-201 I ; state 
assessments have been changing to meet new standards of rigor; 
the relationship among state longitudinal data systems, other state 
systems, EWS and district data systems have been unpredictable 
and in many cases conditioned on previous hardware and 
software choices made for different purposes; and the economic 
downturn has meant an increased focus on cost constraints, often 
accompanied by the closing of programs. These complexities mean 
that the time and costs of developing and implementing an EWS 
can vary dramatically. 

Although there are few “one size fits all” prescriptions, common 
threads tied together the creation, design, and implementation of 
EWS across the country. We learned that EWS are consistently 
the products of vision and teamwork, sometimes among 
unconventional partners. With just a few exceptions, participation 
by districts or schools is voluntary, the result of driven individuals 
seeking something greater for their students. We confirmed that 
EWS are an emerging tool, with most EWS design starting no 
earlier than 2006. Though the field’s nascence translates to a lack 
of formal impact studies, we saw that many EWS are frequently 
re-developed, generally increasing technical sophistication while 
simplifying usability. 

From the policy perspective, several pieces of federal legislation 
with implications for the growth of EWS have been introduced 
or considered by Congress such as the Secondary School Reform 
Act. EWS are also likely to be included in ESEA reauthorization. 



Furthermore, state and federal legislators are prioritizing high 
school graduation in their policy agendas. For example, in 2008, the 
U.S. Department of Education took executive action that requires 
all states to report their graduation rates using a consistent 
formula, and set benchmarks for improvement. Some states 
are considering using on-track measures as part of their next 
generation or school accountability frameworks. 65 

Through these conversations and site visits, we also learned 
valuable lessons about successful implementation of EWS. In the 
best cases, teachers said they felt more effective and equipped 
with better information about which students needed which 
interventions at what time. Educators within school walls and 
nonprofit partners shared they were more targeted in their 
interventions and produced stronger results. District leaders 
felt they had better information about students, so they could 
better allocate precious resources. In some locales, however, 
technological glitches stalled progress. Inaccurate data entry 
produced misinformation. Communications with stakeholders 
was infrequent or non-existent. In summary, we found that 
to be successful: vision and mission matter; technical 
components must be strong; and resources must be 
assembled and maximized, fully engaging stakeholders 
to use data to guide intervention at the scale and 
intensity required. The following provides an overview of our 
findings from our 23 site visits and case studies along these three 
major themes. 
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The Core of Most Effective EWS 



r 



A rich set of knowledge has accrued from local experience over the last seven years. 

Features at the core of the most effective EWS include: 

• Someone or some organization believed EWS were needed and became its champion and shepherd. 

• A leadership team shared the champion's view. 

• End-users had an important role in shaping the EWS. 

• Decisions were made about goals for the EWS both in terms of what it would achieve, and, as a system, it would do. 

• Perfection was not attained in one swipe. EWS evolved over time as roadblocks of many kinds were overcome. 

Two and three years can be necessary. 

• Many EWS began small and learned lessons from pilot efforts before expanding. In other cases, EWS were a separate 
component of larger state or district data systems initially developed for other purposes. They have evolved, building on 
the increasing technical sophistication of the larger data system and the evolving ability to reduce complex information to 
simple presentations. 

• Thresholds and flags were determined based on the ABCs, and a few additional, locally relevant indicators included based 
on local analysis. 

• Student records were flagged when school adults recognized that one or more thresholds were being reached, and those 
students were placed on a support list. Some support lists have 10 students while others may have hundreds. 

• Schools determined their capacity for helping support list students.The size of the list, and availability of help and time, 
forced choices and often caused new partners to be sought, both internally within school buildings and externally from 
the community. 

• Teams of educators and others responsible for support list students matched available interventions with needs — 
or designed new ones. 

• Student records are reviewed frequently — often bi-weekly. In the best cases, teams of educators who share the same 
students shared pertinent knowledge about the students, monitored the effect of interventions, and created new 
assistance plans. 

• As technology improved and data systems integrated, short, simple analytical reports and data dashboards were being 
created that were easy to view, understand, and act on. 

• Educators and other users were supported in understanding and using data, with tools ranging from web-based manuals to 
professional development. Often the most effective support came from coaches. 

• Data was kept and shared about the success of interventions. New efforts focus on understanding the efficacy of and 
determining the best intervention for different students in different situations. 
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I) A COMMITMENTTO SUCCESS 

Sharing the Vision that All Children 
Should Graduate High School, College 
and Career Ready 



attendance and graduation rates and reducing dropout rates. 
Districts and schools are required to have the DEWS in place. 
Although they are not required to use it, estimates show that 
70 percent of Louisiana’s 70 districts use DEWS, and 35 percent 
of the schools are rated as using it well. (Additional information 

is available in the Louisiana 



> 



Early Warning Systems are , almost by definition, promoted 
by strong leaders and advocates in places that have been 
able to make their commitment to educational improvement 
explicit — and for the long haul. 



Case Study.) 



Early Warning Systems are, almost by definition, promoted by 
strong leaders and advocates in places that have been able to make 
their commitment to educational improvement explicit — and for 
the long haul. EWS are not encountered in places where dropping 
out is still considered an acceptable part of the landscape. Barriers 
also exist to the full and successful implementation of EWS. 
Educators may initially perceive EWS analysis as an added burden 
rather than a complementary approach augmenting instruction. 
Counselors and social workers may be overwhelmed by difficult 
caseloads. New technologies present a learning curve for many. 
Financial resources constrain needed coaching and professional 
development. To be able to effectively use attendance, behavior, 
and course-performance data, school-level leadership must share 
a collective vision that graduation rates can be improved through 
the timely and data-driven identification of students who require 
additional supports paired with an organized system to provide 
them. School-based personnel must buy into the concept of 
the expanded data-driven culture. There must be resources to 
build capacity, and the technology systems must be in place to 
provide complex integrated data in short, highly useable, readable, 
and digestible formats.To overcome these obstacles a strong 
commitment to enable all students to succeed is required. 

State, District and School-Level 
Commitments 

Two states have set a high standard for vision and mission by 
explicitly considering EWS as part of the state accountability 
system. In Louisiana, the Dropout Early Warning System (DEWS) 
pioneered in 2004, is the nation's oldest state-level EWS. 



In Virginia, a commitment 
to “graduation for all” was 
demonstrated through a strong 
partnership among the Governor, 
the General Assembly, the State Board of Education, and the State 
Department of Education. Through their collaborative efforts, 
Virginia put measures in place to hold schools accountable for 
meeting graduation benchmarks with a Graduation Completion 
Index, legislated using the State Standards of Learning (SOL) 
accountability and accreditation system. (The No Child Left Behind 
sanctions and supports apply only to Title I schools, and most 
Virginia high schools did not receive Title I.) Now, both Title I and 
non-Title I high schools that do not meet an 85 percent graduation 
rate will be accredited “with warning” or “provisionally accredited.” 
These schools must use the state-developed Virginia Early Warning 
System (VEWS) or a similarly structured, equivalent system. These 
schools must also organize an improvement team, submit a three- 
year improvement plan to the state, and implement it. Virginia 
anticipates that 40 out of approximately 700 high schools will 
receive accreditation with warning/provisionally accredited due to 
the Graduation Completion Index beginning in 201 1-2012 and thus 
be required to use VEWS or its equivalent. 

Several other states have demonstrated their commitment to 
building EWS capabilities and using them to support on-going 
graduation rate and school improvement efforts. South Carolina 
designed a statewide EWS supporting the graduation rate 
improvement goals outlined in the Economic and Educational 
Development Act of 2005 and is moving to a late 201 I launch 
after overcoming technical challenges. The effort will be supported 
with web-based tools and web-based roadmap. Tennessee is also 
moving towards statewide EWS efforts. Texas has not explicitly 
created an EWS, but is moving toward creating the conditions 
under which local districts are empowered to do so.This is 



DEWS is attached to the state 

Two states, Louisiana and Virginia, have set a high standard 
for vision and mission by explicitly considering EWS as part 
of the state accountability system. 



accountability system through 
the school performance-scoring 
rubric that encourages improving 



> 
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especially important in Texas where more than 80 percent of its 
1,000 school districts are small (serving less than 5,000 students) 
and often have limited personnel or technical capacity to design 
such systems on their own. It has recently announced free access 
to two student information systems, which are currently used in 
about half the state’s schools. Texas is also re-tooling its 25-year 
old state performance education information management system 
(PEIMS);the new PEIMS will build in features enabling districts to 
upload data and receive reports of their choice, in a timeframe of 
their choosing. 

Other states have launched pilot programs with the goal of later 
expansion. In 2007, the Washington legislature directed the state 
education department to convene the Building Bridges Workgroup 
(a state workgroup around EWS), initiated a pilot, and developed 
a strong web-based guidebook for EWS teams (please see 
Appendix 2 for additional information). In Massachusetts, a dropout 
prevention commission investigated and reported on policies that 
may contribute to dropout rates.The state is now in the process 
of implementing the commission’s recommendation to expand the 
state’s EWS to all districts and to earlier grade levels. 

In the fall of 201 I, the California Department of Education (CDE) 
launched a model program in nine volunteer districts and 20 
schools, using the National High School Center’s (NHSC) Early 
Warning and Intervention Monitoring System (EWIMS) analysis 
tool, and a similar tool being developed by the NHSC for the 
middle grades. Ten offices across different divisions within the 
CDE contribute to this effort, initially organized by the middle 
grades office as part of dropout 
prevention efforts. The American 
Institute of Research (AIR) and 
WestEd are documenting the 



process as the basis for informing expansion to 50 or more 
districts in coming years. High school, middle school, and district 
personnel have been trained by NHSC and CDE representatives 
on the tool. Schools are provided coaching by state representatives, 
county offices, district offices, and school peers through 
Communities of Practice, regional meetings, and one to one 
support.The CDE plans to further assess how many of California’s 
districts and schools already have early warning systems and 
what they have learned during implementation. They also plan to 
promote the correct use of terms like "truant,” “habitual truant,” 
“chronic truant,” and “chronic absentee” which are defined in 
California’s Education Code. By using these terms in the same way 
it is possible to share lessons learned in attendance data analysis. 

It is also possible to deepen understanding about the influence 
of state and local board policies on attendance improvement and 
dropout reduction, including how the legislatively mandated School 
Attendance Review Boards (SARBs) can most effectively contribute 
to improving graduation rates. 

Within districts and schools, the site visits gave evidence of 
the importance of mission building within the context of clear 
communications and relationships. In Minneapolis, mission-building 
work with the students themselves was core to their success. 

The District’s Check and Connect monitors emphasized that 
educators have to get to know students “outside the context of 
the data.” While the attendance and course performance data 
were important in identifying students, monitors found building 
trusting relationships to be essential to moving students forward 
in their studies. “You have to see what their motivations are and 
move from there,” one Check and Connect monitor said. Mentors 
focus on positive aspects of a student’s record and build confidence 
through encouragement before discussing shortcomings. So far, 
data show that the Check and Connect system has improved 
student retention, attendance, and completion rates. On a national 
level, Check and Connect is highlighted as a successful effort in 
the U.S. Department of Education’s What Works Clearinghouse. 

A similar emphasis on building a sense of mission around EWS 
through establishing relationships and building trust was evident in 
the work of Big Brothers Big Sisters of Eastern Missouri with Cape 
Girardeau and St. Louis students and schools and in the Diplomas 
Now secondary school transformation efforts in 14 school districts 
across the nation. (Please see case studies on each for additional 
information.) 



> An emphasis on building a sense of mission around EWS 

through establishing relationships and building trust was key 
to success. 
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Big Brothers Big Sisters of Eastern 
Missouri - A Focus on the ABCs 



Big Brothers Big Sisters of Eastern Missouri (BBBSEMO) and their local school districts have created a unique, replicable and 
outcomes-oriented partnership that leveraged an early warning system (EWS) to improve the lives of children. Following an 
intense strategic planning process in 2005, BBBSEMO revised its vision and mission statements and set out to turn their big 
vision into meaningful, measurable and tangible outcomes for local children. 66 

Focusing on the ABCs 

BBBSEMO began by identifying a partner who was willing to innovate: Cape Girardeau Public Schools, where one of four 
students failed to graduate on time. 67 BBBSEMO asked, “What results must we deliver in order to be considered a high value 
partner?” Over nine months, BBBSEMO, the superintendent, principals, and Southeast Missouri State University educators 
developed ABC Today!, a program focused on tracking students’ Attendance, Behavior, and Classroom success as the 
first step in returning them to the graduation path. The agency planned to devote one year with the school district collecting 
this information — manually — and identifying patterns, policies, and interventions to guide their later work. Long-standing 
relationships enabled BBBSEMO to design data access procedures and obtain parent or guardian consent to meet FER.PA and 
district requirements. 

Workers began to quickly uncover unnerving new information about the children they had served for years: a second grade girl 
had been absent for 28 days due to chronic head lice and had failing grades in reading and math; a fifth grade boy had missed 38 
days and had accumulated 13 discipline referrals that began after his father returned from prison. The organization decided 12 
months was too long to wait for fine-tuning a process, and began “refueling in mid-air.” BBBSEMO targeted students who were 
flagged for increased interventions and soon saw results: attendance and course performance improved, and the number of 
tardies and behavior infractions decreased. Stakeholders are convinced that data helped — “We don’t have to fight over ABC; we 
know these are key performance indicators so we just have to work together to improve them.” 

Replicating and Improving What Works 

After the successes of the first year of data-driven interventions in Cape Girardeau, BBBSEMO was invited to replicate the 
model in St. Louis Public Schools. A Director of Impact was placed in the five schools and meetings were held — monthly with 
principals and quarterly with district leaders — to review data, identify trends, create intervention plans and examine outcomes. 
The BBBSEMO staff met weekly to identify successful interventions and retool less successful efforts, and to set incremental, 
school-specific goals, regularly holding “blow it up” conversations — to “inflate” a good idea or “pop” a strategy that was not 
producing results. Partners learned to balance the frequency with which data was examined against the time needed to see 
results. 

Partnerships Build Even Better Results: Within four years, BBBSEMO’s work was endorsed by Civic Progress, an 
organization of St. Louis CEOs and leading executives. Wells Fargo Advisors stepped in with financial, technical, and human 
resources, greatly advancing the agency’s capacity by building systems to electronically transfer data between district and 
BBBSEMO databases, while retaining the early emphasis on easy, fast, and convenient processes. 



Widening the Circle of Engagement: One result of the attention on outcomes has been a focus on results for all. 

When BBBSEMO first began using student data, staff looked only at red flags and acted on problems. But, as one staff member 
remembered, “it occurred to us that we should do something with every piece of data.” The staff decided to broaden attention 
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to students who had no early warning indicators, enabling active reinforcement of good behavior and student performance 
across the board. As an unforeseen result, relationships were built with the parents of well-performing students and the number 
of volunteers and positive influences engaged with the school expanded. Now, according to BBBSEMO,“We celebrate or 
intervene on every piece of data we have.” Even that requires work. BBBSEMO is working with the school districts to develop 
standards around celebrations, including agreement on what qualifies as success and how to identify it.This collaboration 
and partnership is helping to make the culture of intervention and celebration a broader part of district and school culture. 
According to one district leader, “If our students are going to succeed we must create connections with the community 
and foster long-term commitments that provide support and encouragement to our families. Our school district is deeply 
committed to expanding our partnership with BBBSEMO and strengthening ABC Today! because it puts good hearts and data 
side-by-side.” 

Lessons Learned 

Relationships Matter in Effective Partnerships: According to one district leader, “we have seen first-hand how... 
collaboration can improve the educational opportunities and outcomes for our kids.” The pre-existing relationship with the 
community eased concerns around data privacy and access, and made parents comfortable with the role the organization was 
playing in their schools. The close relationships between BBBSEMO, the Cape Girardeau and St. Louis School Districts enabled 
educators and BBBSEMO workers to more effectively identify the causes of absenteeism and problem behavior, and craft 
effective interventions. 

Keep it Simple: The flexibility of all parties in adapting the data system and re-evaluating student trends enabled an effective 
system. The constant quest to keep things simple and the traffic-light warning system created a reminder of the importance of 
usability when presenting educators and volunteers with complex data. 

Moving Forward: Continuing its organic evolution, the BBBSEMO is considering replication opportunities locally and 
nationally, finalizing the development of a new web-based ABC Today! Intelligence Tool and launching a whole-school pilot reform 
model called ABC Today! School. Reflecting on past successes and guideposts for the future, BBBSEMO’s CEO remarks that the 
most effective change agents of all were school administrators — accountable, adaptable, individuals who provided a vision and 
supported a culture of high performance that enabled the agency’s efforts to thrive. The CEO also shares: “Collecting the data 
was really not the hardest part, it was building the cultures of trust, accountability, discipline, innovation, and adaptation that 
drive the positive change.” 



2) TEC HIM 1C ALC OMPONENTS 
MUST BE STRONG 

Data and System Design Can Promote — 
or Hinder — Student Success 

In conversations with EWS implementers and users across the 
county, it became clear that the design of EWS, and the data within, 
mattered. Before an educator can deploy resources effectively to 
the students who need them, he or she must be able to accurately 
identify which students are on 
track and which are off track to 
graduate. In order to appropriately 
leverage the utility of data, data 
must first be collected accurately, 



then analyzed into usable and actionable information — and 
educational entities must know how to do so. Schools and districts 
at the vanguard have created easy-to-understand dashboards and 
“stoplights” for this purpose, simplifying data from multiple, often 
complex sources into resources teachers and other school-based 
adults can use. 

The issues of data usage, by whom, and for what purposes, 
while protecting privacy and ensuring compliance with existing 



> ““| Before an educator can deploy resources effectively to the 

students who need them, he or she must be able to accurately 
I identify which students are on track and which are offtrack 
to graduate. 
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regulations, can combine to either overwhelm users or streamline 
interventions. Here, conversations with education leaders 
illuminate both the best practices and the cautionary tales about 
how to leverage data and system design to promote student 
success at the state, district, and local level. (Please see Appendix 2 
of this report for a listing of publications related to privacy issues 
in an education setting.) 

Ensuring Accuracy at All Levels of 
Educational Organization 

Misinformation can stall progress. Inaccurate data produces 
incorrect recommendations for intervention, citing too many, 
too few, or incorrect students as “off track” and for the wrong 
reasons. Accuracy is necessary at all levels of the data chain, 
and most often this falls on the shoulders of school-level data 
clerks, data operators, attendance specialists, registrars, teachers, 
counselors, and social workers at the student or classroom level, 
and assistant principals, deans, and academy leaders within the 



school. In South Carolina, for example, while teachers enter grade- 
book data, schools have a state-funded data clerk who enters 
free- and reduced-price lunch data, discipline data, career and 
technical information, out-of-school and extended time program 
enrollment, at-risk status, and interventions. State funding that 
defines responsibilities for the data clerk position places a premium 
on correct data being entered in a timely fashion, and provides an 
accountability avenue when there are inaccuracies. 

In most locales, reporting positions are determined by local 
leaders and local allocation, particularly for reporting of 
behavior infractions, participation in interventions, outcomes of 
interventions, and out-of-school-time data.This ad-hoc organization 
of data entry can lead to less accountability and accuracy. To 
address this challenge, some early warning data systems, such as 
the Metro Nashville system, have built in validation levers with an 
opportunity for data correction. 



Lessons from Nashville: Make it manageable. 
Make it meaningful. Make it matter. 



Metropolitan Nashville Public Schools (MNPS) is a large, diverse city/county school system serving 78,000 middle Tennessee 
students, 70 percent of whom are eligible for free- or reduced-price lunch. The District is a blend of both low-performing and 
high-performing schools. Its efforts to build and implement an EWS system are situated in a multi-year improvement effort 
called Nashville Achieves, which involves broad-based community participation, including the Mayor’s Office, City Council, 
business and nonprofit organizations and input from over 200 community leaders involved in transformational leadership 
groups advising the direction and content of reforms. In designing and implementing their EWS, MNPS exercised leadership and 
identified help where needed. They invested heavily in rollout and implementation of their system, integrating it thoughtfully with 
instructional reforms. Their system is one manifestation of the district’s cultural shift toward data-driven improvement and grew 
out of the superintendent’s advice for a data system:“Make it manageable. Make it meaningful. Make it matter.” 

Getting the System Right 

MNPS received technical support from the Everyone Graduates Center at Johns Hopkins University to develop their EWS 
triggers. Together, they analyzed longitudinal district data in the context of state and local policies, practices, and conditions. 
Currently, attendance, behavior, and course performance are flagged, with thresholds of less than 85 percent attendance, more 
than five offenses for behavior, such as suspensions, or less than a 70 percent average in core classes. Absence reports (including 
excused, unexcused, in- or out-of-school suspension) are differentiated in the reporting system. These triggers will be reviewed 
each summer to make sure that they accommodate most recent trends in data. Previous years can be viewed so that individual 
student trajectories are evident over time. 
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Lessons Learned 

• Use Existing Resources: MNPS’ EWS was designed in-house and has grown organically over the last two years as a 
tool for instructional improvement, combining increases in technical complexity with user friendliness and implementation 
support for leaders and teachers. 

• Keep it Simple: Sophisticated navigability means that the Director of Schools (Tennessee’s term for Superintendent) can, 
with only three clicks, see each school’s overview.The easy-to-use educational data warehouse is refreshed nightly and the 
system itself is accessible. 

• Provide Training and Support: As the EWS fully moves out to the schools, a new unit is being formed within 
the central office to guide EWS implementation at a granular level. Previously principals and leadership teams became 
acquainted with system features in a gradual two-year process that created knowledgeable site-based mentors and 
advocates. By 2010-201 I, observers perceived that most principals were comfortable interpreting the data now available 
in increasingly clear, clean dashboards and over 130 reports. Beginning in 201 1-12, twelve central office data coaches and 
one oversight coordinator were made possible by Race to the Top funding.They will be deployed to each of the 12 school 
clusters, with priority given to high-needs schools. The data coaches are charged to build a culture among teachers and 
counselors for understanding student-, classroom-, and school-level EWS data, and using it to guide intervention efforts. 
They also augment a sizable corps of instructional coaches and reflect a district-wide investment to build human capital 
through professional development and coaching. 

Challenges 

Challenges remain and will be addressed in the same way that the system was developed with ingenuity, creativity, and patience. 
Some schools are more welcoming of data coaches than others. There is not yet an easy interface with state data systems. 
Teachers do not uniformly use the electronic grade-book.The data system does not currently identify overage or under-credited 
students effectively. Interface with transportation and financial systems will help the district understand and improve student 
mobility and cost issues.A FERPA compliant interface with external community agencies will help MNPS assist its students with 
a wider range of interventions. 



Finding a Common Language at All Levels 
of Educational Organization 

As states work to build and strengthen state longitudinal data 
systems and EWS, they must work to integrate existing systems 
from technological and semantic perspectives. However, there is 
an almost pervasive and unresolved tension at all levels between 
“local control” and the standardization that is necessary for good 
data systems. Many are working in good faith to find acceptable 
solutions. Nationally, several groups are working to evolve 
frameworks of technology standards for database compatibility, 
interface, and data transport.These frameworks include: 

• Schools Interoperability Frameworks (SIF), an industry 
initiative working toward a set of rules and definitions to 
enable software programs from different companies to share 
information across platforms. 68 



• The NCES Handbooks, a set of guides that provide 
standardized business definitions and coding schemes that are 
necessary if systems are to be developed that draw from and 
compare multiple sources. 

• Ed-Fi, a free data standard system developed by the Michael 
and Susan Dell Foundation, aimed at K- 1 2, is vendor-neutral, 
modifiable and capable of connecting multiple databases while 
keeping operational systems, compliance reporting systems, 
and budget systems separate as desired. Five pioneering 
states — Colorado, Delaware, Louisiana, Tennessee, and 
Texas — were the first to implement.A developing 

Ed-Fi Alliance of users is anticipated to provide input into 
future improvements. 



EWS designers often encounter unexpected challenges unrelated 
to information technology systems per se. In work that began in 
2006-2007, through the Statewide Tools forTeaching Excellence 



28 



Part 2: Lessons from the Field 




(STTE) initiative funded by the Michael and Susan Dell Foundation, 
and building on other Texas efforts that began as early as 2003, 69 
collaborators from five Texas districts and researchers from 
the regional education laboratory, REL Southwest at Edvance 
Research encountered a key stumbling block in designing an 
EWS that worked for all: different coding systems were in place 
in the different districts. Three used a pass/fail code for courses 
on a semester basis; one used both a pass/fail code and numeric 
grades on a semester basis; and one district used a pass/fail code 
but recorded credits on a yearly rather than semester basis. As 
one Edvance leader commented, “it was quite a process to come 
up with a common coding structure across the five districts due 
to tremendous variability in coding, weak data quality, and lack of 
local familiarity with data.” Dialogue led to solutions that were 
later applied in the larger Texas Consortium on School Research 
(TCSR) 70 which now engages the original five and 28 otherTexas 
districts (together serving 37 percent of the state’s students) in 
examining and refining the applicability of the Chicago On-Track 
indicators in settings far-removed from the urban Midwest. (See 
Case Study on Chicago for more information.) In the 2011- 
2012 school year, TCSR districts will move to a similar reporting 
format using course grades, and for similar time periods. Common 
understandings developed through long-term collaboration are 
fruitful for the success of EWS. Boston Public Schools and its 
EWS partners encountered similar challenges. Schools had no 
common criteria for courses with common names. Some had the 
same content; others did not. A district course catalogue with 
standardized names and content descriptors was necessary to 
enable accurate analysis of students’ transcripts, 

GPAs, course passing, and credit accrual. 



Reducing the Burden of 
Too Much Information 

As the technical interfaces among different databases have 
improved, more information has become accessible for different 
constituencies within a state, district, or school. In some cases, 

EWS that used excel spreadsheets five years ago can now produce 
voluminous reports extracted from data warehouses. This can be 
overwhelming to inexperienced data users, or those for whom 
data analysis was not previously a priority or need. Too much 
information can reduce buy-in and ownership at the school level. 

A number of solutions have emerged to reduce the burden of too 
much information. In Houston, designers of the Dropout Prevention 
Early Warning System (DPEWS) used the centering question, “How, in 
an environment of data-driven decision making, can we reduce ‘analysis 
paralysis?” As a result, they were able to produce reports around 
a few key research-based predictive indicators, in which end-users 
could be confident. In Louisiana, the DEWS now automatically delivers 
timely reports over email to administrators and teachers, eliminating 
the need for end-users to query the system. When Massachusetts 
designed its original state system, it intentionally attempted to keep the 
number of indicators minimal. In districts such as Nashville, where over 
130 customized reports can be available, designers are considering 
producing a smaller, standardized set of those that have been found 
to be most useful. In Memphis City Schools, in the early stages of 
developing an EWS, the superintendent sends out a monthly two- 
page report, “From 30,000 Feet” which brings to principals’ attention 
relevant district-specific findings produced by the Research and 
Assessment Department. 

Across the country, role-specific data captured in EWS is 
increasingly presented for principals, teachers, students, and parents 
in web-based dashboards and portals. Color-coded “support lists” 
are provided for teams of educators that outline which students 
are struggling to show up, which students misbehave, which 
students are struggling academically, and who is receiving which 
intervention with what result. In many cases, data systems can 
produce short individual profiles of on- and off-track students with 
contextual detail and/or summaries of in-coming students sorted 
by their level and type of off-trackness, as in Chicago, Dallas, and 
Philadelphia. A number of commercial systems are also available 
as part of student information systems or as add-ons that can 
enable individual schools or districts to pull together existing data 

and create actionable EWS data 
reports at the student, teacher, 
classroom, and school level. 



> "How ) in an environment of data-driven decision making, can 
we reduce ‘ analysis paralysis ’?" In many districts, color-coded 
"support lists" are the answer. 
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Achieving Buy-In through Local Input at 
All Levels of Educational Organization 

The three most important indicators — attendance, behavior, and 
course-performance — are well established, along with widely 
applicable thresholds. Nonetheless, sometimes there is resistance 
to adopting EWS developed by 
others. EWS designers have found 
that enabling local customization, 
while maintaining core research- 
based components, can increase 
both buy in and utility. For example, 
in Texas Diplomas Now sites, Franklin City, Virginia, and Knoxville, 
Tennessee, EWS designers perceive that it was the inclusion and 
ready access to the results of formative assessments that spurred 
teachers' buy-in to the EWS.The relevance of the system to daily 
instructional work became immediately evident. 

In Knoxville, Chattanooga, and Nashville, EWS designers found 
that EWS support increased when focus group input was sought 
from administrators and teachers during the development and 
refinement stages. In Chattanooga, for instance, the Division of 
Accountability and Testing developed the early warning system 
relying on feedback from internal stakeholders. “Data meetings” 
with educators were held in 2005-2006, first with high schools, 
then by regions, and eventually including elementary schools. 
Existing data was shared and discussed. Central office supervisors, 
principals, Exemplary Educators (state-assigned veteran principal 



coaches), and leadership teams were asked, “What would be 
helpful to you in raising student achievement? What would be 
helpful in improving the graduation rate?” Tennessee, in designing 
its statewide system, sought input and recommendations for design 
specifications from a diverse set of school districts.They later 
asked them to serve as pilot sites. 



Participation in setting thresholds can also increase user buy-in. 

In Texas, district collaborators and Edvance researchers defined a 
three-step process similar to that used in Chicago: create a support 
list of students from eighth grade records; monitor students’ 
progress on indicators at each grading period; and establish a ninth 
grade On Track indicator. Because they had little empirical data 
on the best absolute value of the cut-offs in their own setting, 
they established defaults that can be reset if local districts believe 
this is appropriate. Researchers and district participants asked 
the question, “What should the thresholds be in our situation? 
When we review data, are we reviewing data that indicates student 
problems, or does it indicate system problems in recording 
data?” In Virginia, sites that use theVEWS will be enabled to set 
thresholds-if they are more stringent than the state designated 
and validated ones. The state recommends that sites have a year’s 
experience withVEWS before working on this nuance. 




EWS designers have found that enabling local customization, 
while maintaining core research-based components, can 
increase both buy in and utility. 



Keeping Freshman and Others 
On Track in Chicago 



Behind the Scenes 

Chicago Public Schools (CPS) is the nation’s third-largest school district, with 407,000 students and a long-standing and serious 
graduation challenge: in the mid- 1 990s, as in many urban districts at the time, the graduation rate was 46 percent. Numerous reforms 
have been implemented by superintendents since then, with the net effect that by 2010 the graduation rate had risen 20 percentage 
points, 71 a great improvement but still far below goals. 



In a strong partnership with CPS, the Consortium on Chicago School Research (CCSR) began analyzing longitudinal CPS 
data in the late I990’s. In 2005, CCSR released a report identifying the major factors that predict CPS ninth-graders’ high school 
completion: failing no more than one class per semester and completing sufficient credits to be on track for grade-level promotion. 
These factors were combined into the “Freshman On-Track” rate, and proved more accurate at predicting at-risk students than 
standardized test scores or student demographics. In 2007, the CCSR confirmed that using the Freshman 
On-Track rate, along with GPA and number of course failures could accurately predict a student’s graduation or dropout 
status 80 percent of the time. On-Track freshmen were almost four times more likely to graduate within four years than 
off-track freshmen. 
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Getting Started 

In 2008-2009, CPS implemented a two-pronged initiative translating research into action. District-wide, 

high schools were informed at school opening by their Area Instructional Officers and by memos that actionable data for incoming 
ninth-graders was available electronically in a “watch list” that included eighth grade attendance and course performance. Later in the 
year, “success reports,” distributed every five weeks at the end of a marking period, flagged students with attendance issues or a D or 
lower in core courses. Principals were urged to organize their schools to use this data to identify and intervene with students who 
needed extra help, a recommendation given clout by the inclusion of “On Track” indicators for ninth graders in the framework for 
principals’ accountability. 

In the same time period, a one-year grant from the Bill and Melinda Gates Foundation enabled creation of intensively-supported pilot 
Freshman On-Track labs in six high schools, with the district’s intention of taking lessons learned to scale. Two paid coordinators 
were hired for each lab school, and they, with support, training and protocols from the area offices, assisted school administrators, 
teachers and counselors in understanding the data, using it, and designing and implementing innovative intervention plans for students. 

While all six of the labs received equal resources, there were varying degrees of success. Collaboration among administrators, 
teachers, and researchers was crucial to the use of data. Schools in which there was little support from administrators, or in which 
resources were spread too thinly over too many students did not achieve the success of other schools. Schools with high teacher 
involvement yielded the most success. At the end of the school year, lessons learned were compiled and analyzed, resulting in 
“Freshmen on Track: A Guidebook to F-lelp You Keep Your Freshmen on Track to Graduation." 

Keep Moving, 2009 and Forward 

In school year 2009-20 1 0, the Freshmen on Track guidebook was provided to all high schools, over the web, and on CD. Schools 
continued to receive watch lists and success lists. Misconduct data, both aggregated and student-level, was available in a separate 
report, the “Student Profile,” and the level of detail about what should be accessible to whom continued to be under discussion. 
End-of-semester reports were added, focusing on credit accrual and need for placement in credit recovery to get back on track. 
Work on using the On Track indicators became more intentional; some area offices implemented coaching systems to provide 
targeted supports to schools; some had monthly meetings; and all held performance management sessions in which interventions to 
keep students on track were discussed. Teachers utilized lists to shape their classroom activities, and schools used them to manage 
extracurricular participation and wraparound service provision. Some schools put in extra effort; those in which data (other than 
attendance) 72 were inputted daily were able to extract information and reports on a daily or weekly basis. Recognizing the value of 
additional support for using data and guiding interventions, some of the principals of the On Track lab schools were able to allocate 
funds to continue to support a coordinator even though grant funding had ceased. 

By 20 1 0-20 1 I , the On Track system, although still variably implemented, had become institutionalized in the district and expanded to 
include tenth graders, based on a set of tenth-grader specific indicators; accountability for their On-Trackness was added to the High 
School Score Card. As the school system reorganized in 201 1-2012, with networks rather than area offices, the work of providing 
help to schools shifted; schools that are part of external networks (for instance, the CCSR’s College Success Network) are further 
supported and coached in data use and interventions by external partners, just as some internal networks also provide help. 

Impact 

By several calculation methods, the On Track system has been successful. CPS data show that the freshmen On-Track lab schools 
have a 76. 1 percent “on track” rate, exceeding rates before implementation and higher than the district average of 63.9 percent. 
District wide, with data from CCSR, the On-Track rate has risen approximately 10 percentage points, from a baseline of 56 to 59 
percent in 2004-2007 to 69 percent in 20 1 0. 73 
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Moving Forward During Tight Fiscal Times 

Although recent cuts to data personnel and other services have reduced the degree to which CPS can support schools, providing 
student data is continuing. CPS believes that in the long run, investment in the On-Track system saves money, as it enables educators 
to aid at-risk students earlier and to avoid costly supports later.The On-Track system was scaled to serve 60,500 ninth and tenth 
grade students at 107 high schools, and is slated for expansion to the middle school level and possibly eleventh grade during 2012- 
20 1 3. The first cohort of students monitored through the On-Track system will graduate in 20 1 I -20 1 2, enabling impact studies.The 
district also plans to analyze different elements of data and effectiveness, including: which interventions are working best for students; 
how, longitudinally, students move on and off track; what grade distribution looks like, and how this affects student performance. With 
clearly documented rising graduation rates, the challenge ahead is to make sure that more high school graduates graduate well- 
prepared for college and work. ACT scores, a benchmark for college readiness, have risen one percentage point in the last decade 
while participation has increased 10 percentage points. 74 The goal is to achieve even higher levels. 



3) RESOURCES MUST BE ASSEMBLED 
AND MAXIMIZED 

In our site visits, we found that data alone are not sufficient. 

To turn data into results, appropriate interventions had to be 
assembled and human capital had to be available to provide these 
interventions. Further, states, districts, and schools are under 
pressure to produce substantive performance growth, while, in this 
economy, resource constraints are all too real. In the face of the 
need to simultaneously maximize education dollars and student 
performance, states, districts, and schools are being creative with 
leveraging and efficiently using resources inside and beyond the 
schoolhouse and district walls. In some instances, improved internal 
alignment has capitalized previously under-utilized or misdirected 
resources. In certain resource-constrained instances, external 
partnerships were essential to success. 

In our conversations with education leaders across the country, 
we found that these resources exist, both inside and beyond the 
schoolhouse walls. Conversations with EWS leaders revealed 
that partnerships of all kinds offer new perspectives and produce 
results at the state, local, and community levels. Design and data 
teams brought in unlikely partners to provide fresh ideas. Teachers 
were organized into more effective teaching teams; educators made 
better use of existing support personnel like counselors and deans 
of discipline; and community organizations provided services like 
mentoring and tutoring or more intensive wrap around supports 
for a range of high needs students at all levels. Nonprofits helped 



solve the scale problem by supplying a second shift of adults when 
schools find that they have hundreds of students with off-track 
indicators in need of additional supports. Supportive professional 
development, coaching, and networking were key to the process. 
Below, we share a few examples we encountered of resources that 
were critical to EWS implementation and how to best leverage all 
available resources to keep students on track for success. 

Share Knowledge, Needs, and Ideas 

EWS users are learning from each other across state, district, and 
school boundaries. In Diplomas Now schools, teacher teams meet 
weekly or bi-weekly to review and discuss EWS “support lists” 
assembled by a coordinator. Once student-by-student, group, and 
whole school challenges were identified, team members put their 
heads together to arrive at suggested interventions, designate 
a responsible team member, and assess progress two weeks 
later. Teams of near-peer mentors and tutors from City Year (a 
national service organization), social workers/site coordinators 
from Communities in Schools (an integrated student services 
organization) and a school transformation facilitator from the Johns 
Hopkins Talent Development Secondary School reform program, 
complement the teachers and other school personnel on the 
team. They provided both insight into the challenges students face, 
and additional person power to help implement and monitor the 
proposed solutions. Early adopting EWS states and districts are 
also sharing their experiences with others.The Nashville EWS 
team, for example, presented what they 
had learned about EWS design at several 
meetings, including a statewide focus group 
meeting for the design of the state system, 
a national data meeting sponsored by 



> Data alone are not sufficient.To turn data into 

results , appropriate interventions must be assembled 
and human capital must be available to provide these 
interventions. 
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National Center for Education Statistics (NCES), and in a visit to 
Memphis City Public Schools. 



Educators and technical experts also serve as mutual resources, 
first developing and then fine-tuning EWS to most effectively meet 
local needs in cost effective ways. In districts, regions, and states, 
EWS have been designed by a variety of technical experts.These 
include state and district programmers and IT experts, technical 
experts in regional education service agencies and regional 
education labs, software entrepreneurs, corporate partners, and 
sources such as the National High School Center. In Columbus, 
Ohio, Nationwide Insurance enrolled executives and technical 
staff to help spearhead district efforts to build an EWS.They 
then spun-off a nonprofit organization to help other districts do 



the same. In Cape Girardeau and St. Louis, Missouri, Wells Fargo 
Advisors contributed financial, technical, and human resources to 
fine-tune and implement the technical 
aspects of the Big Brothers Big Sisters of 
Eastern Missouri scale up from manual 
to electronic data collection. In Knoxville, 
Tennessee, after the Board of Education 
articulated visionary goals for student 
outcomes, the Chamber, which sought to 
have the best workforce in the country, 
spearheaded formation of a joint business/ 
school district team and, with foundation support, secured an 
experienced executive to develop a district data system that would 
answer the questions related to return on investment and resource 
allocation. With the core of the data system built in the first two 
years, EWS functions will be added in 201 1-2012. 

In Virginia, when educators faced capacity constraints in developing 
the Virginia Early Warning System (VEWS) technology tool, they 
reached out to the National High School Center, with whom 
they had previously collaborated. As a result of this partnership, 
Virginia transformed simple spreadsheets for single districts 



> 



Resources exist , both inside and beyond the 
schoolhouse walls. Local and national nonprofits can 
contribute essential wraparound services. Educators 
and technical experts can serve as mutual resources. 
Corporate partners can contribute technical expertise. 
Research institutions can guide system design. 



Lessons from Knoxville: Big Vision. Big Results. 
When Schools and Businesses Align 



Knox County Schools (KCS) serves 57,000 students, 40 percent of whom are eligible for free- or-reduced-price lunch. This district 
has some of the state’s best, as well as lower performing schools. Through its five-year strategic plan, entitled Excellence for All 
Children, Knox County Schools set ambitious goals to prepare its students for college, for careers and for life. To address educational 
disparities, the Knox County Board of Education set a tough challenge for their community: 1 00 percent of entering ninth graders 
will complete high school in four years, 90 percent of those will graduate with the regular diploma, 90 percent of those who graduate 
with a regular diploma will have taken the ACT exam, and 90 of those who have taken the ACT will achieve a composite 21 or higher. 



A Perfectly Aligned Partnership 

The Knoxville Chamber, which in the summer of 20 1 I was named national Chamber of Commerce of the Year, set the challenge 
that Knox County and adjoining Oak Ridge would have the best workforce in the country as well as be a beacon for new business. 
To reach this goal, they asked KCS what was needed to achieve the high goals that had been set.Their answer was simple: funding. 
The business community responded to the call, asking questions like: “What should we fund? How will we know what is working? 
How will investment outcomes be measured?” They also got to work. With the district’s permission, MBA interns discovered these 
questions couldn’t be answered because there were 20 separate data repositories. They found that KCS had a great deal of raw data 
insufficiently integrated to answer tough questions about strategic investments. 



The conditions were right for a partnership.The new superintendent was familiar with the value of good data systems from a 
previous role. The business community knew the value of education and possessed strong technical expertise to help with solutions. 
Mutual commitment grew to build a basic integrated data warehouse. A retired CEO stepped in to oversee system design. Local 
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philanthropies committed funds. A technology system from a college bookstore supplier that had powerful querying tools which 
were not operating-system dependent, (necessary because KCS had many different makes of computer) was identified for use and 
later purchased with school system funds. 

Getting to Work 

The superintendent determined that the system should enable viewing student progress longitudinally. Elementary and middle school 
teachers and administrators should be able to look at middle and high school data on elementary school graduates. Their high school 
counterparts should be able to look backwards at students’ performance in middle and elementary school. Focus groups of principals, 
teachers, and counselors were convened to advise on what was needed and in what format. From there, the education and business 
community designed a comprehensive and flexible system that combined a vast amount of historical information. This included 
financial, academic and demographic profiles, state assessments and ACT scores, daily grades, daily and period attendance and tardies, 
daily discipline, enrollments, and mobility among schools. In December 2009, just 1 8 months after inception, the first Education 
Management Information System (EMIS) was unveiled in Knox County. By mid-year 201 I, six years of data had been loaded. 

Lessons Learned 

They Needed to Learn More. Cost-benefit analysis showed that the lowest performing schools were receiving the most 
resources but needed help in choosing and targeting interventions to help the right students most effectively. Educators requested 
more instructional data. In 20 1 I , outcomes of a district-designed formative assessment were added. Interest in these results was 
viewed as critical to developing educator ownership and use. Later that same year, a new reporting tool was released to all teachers 
and principals that assisted them in navigating through the data maze to an individual student profile. With just three clicks of a 
mouse, easily readable, color-coded, two-page profiles appeared. 

The Partnership Continues. The Chamber has not strayed from its original goals. It has continued to press forward with the 
local school district to grow the best-prepared workforce in the country.Through its Workforce Development group: 

• A gap analysis was carried out of present and future workforce needs and how area schools and institutions of higher education 
are meeting those needs; 

• 100 community leaders committed to spreading the word about education and workforce needs with an “Education is a 
Community Issue” campaign and presentations. Materials and a video were developed to support these efforts; 

• 10 or more companies support 100 teachers per summer in once-a-week summer lunch and learns, especially in science and 
math related occupations, giving educators the opportunity to internalize workforce expectations for both soft and hard skills 
and take these back to the classroom; and 

• The business community joined with the superintendent to write a new strategic plan. A Chamber executive also joined the 
KCS implementation team, serving on the superintendent’s leadership team and often spending a day a week in the district. 

Moving Forward 

Latest results from the Tennessee assessment system (from the 2010-201 I school year) show that district scores have continued to 
rise. But the work is not over. In the 201 1-2012 school year, there are plans to add an EWS component with fifth and eighth grade 
flags, and ninth grade indicators. Improvements will also enable data for transfer students to be accessed within a day of registering 
and fuel a study of school feeder patterns and their costs. Partners will continue to load longitudinal data until 1 5 years of data are in 
the system, making it possible to understand historical trends. 
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In several large districts including Boston, Chattanooga, 
Chicago, Jacksonville, and Philadelphia, teamwork between 
public education funds, philanthropists, local educators, 
and researchers were essential to EWS development. 



Public Schools (BPS) longitudinal data, 
and establish “at risk” thresholds for 
the 52,000 students in BPS. 



into a complete, consistent system that would merge state and 
local data.This system imported and exported data back to the 
schools without duplication and made it useable by even the most 
resource-constrained districts. Diplomas Now is working with 
SchoolLoop to develop a portable plug-and-play EWS for schools 
in districts where one is not available. 

Strategic Partnerships Maximize Impact 

Leveraging resources helps everyone to produce results. In several 
large districts including Boston, Chattanooga, Chicago, Jacksonville, 
and Philadelphia, teamwork between public education funds, 
philanthropists, local educators, and researchers undergirded EWS 
development. In Philadelphia, the Philadelphia Education Fund, 
the School District of Philadelphia, the William Penn Foundation, 
and Johns Hopkins University researchers combined efforts 
to conduct some of the pioneering research on early warning 
indicators and integrate the indicators into the school district’s 
student information system. As a result, teachers and parents could 
have access to their student’s early warning indicator data, and 
then pilot and develop comprehensive intervention systems at the 
school level. The Boston Public School System collaborated with 
the Boston Plan for Excellence (BPE), Johns Hopkins researchers, 
ACHIEVE, and later the Parthenon Group, to analyze Boston 



In Chattanooga, the successful 
implementation of EWS was part of 
an overall, decade long improvement effort made possible through 
the strong superintendent, a committed public education fund, a 
dedicated local business and philanthropic community, investment 
funding from the Carnegie Foundation, and the hard work of 
principals and teachers. In Chicago, the Consortium for Chicago 
School Research, working with the school district, conducted 
research on high school early warning indicators. They developed 
an on-track measure for ninth graders, which was subsequently 
incorporated into high school accountability indexes by the 
school district and became part of principals’ annual review by 
their supervisors. This practice was later spread to Dallas. With 
funding from the Gates Foundation, the Chicago school district 
then developed data reports for high schools that provided early 
warning indicators for incoming freshmen and piloted response 
systems. In Dallas, support institutions including the Michael and 
Susan Dell Foundation, the Gates Foundation, the Community 
Foundation of Texas, and local corporate and nonprofit partners, 
assisted Dallas Independent School District (ISD) in moving 
beyond its first phase on-track indicator system to initiating a 
second phase College Readiness Indicator System (CRIS), as 
part of a new five-district network supported by the Annenberg 
Institute at Brown University and the John W. Gardner Center at 
Stanford University. 
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The Franklin City, Virginia school district (a “division” in Virginia 
terms) served as one of four districts in which the state 
department not only piloted the EWS developed by the National 
High School Center, but also the state improvement process 
organized around the EWS. Franklin City is a small urban district 
with a high poverty rate, 57 percent graduation rate, and two of 
three schools already in sanction. Education consultants from the 
Virginia Department of Education and the Mid-Atlantic Equity 
Center worked with the district to establish a team and create 
and implement the required improvement plan. The team focused 
on building the EWS locally, using the technology tools provided 
along with locally designed triggers. For attendance, the trigger 
was the fourth absence; for discipline, it was either the fourth 



in-school suspension or the second out-of-school suspension; and 
for academics, the trigger was failing (a D or F) in one or more 
courses at the six-week interim grading period. Students were 
divided into three tiers, with one, two, or three indicators. Thirty- 
day plans were written for each student with an indicator.Those 
with three indicators were required to attend a meeting with 
the associate superintendent as well as weekly meetings with the 
school team to review their “I Can Do It” plan to get back on the 
graduation path. Franklin City will use the EWS and the process 
laid out in the improvement plan beginning 20 1 I -20 1 2, as the 
state will at the same time move the EWS process into additional 
sanctioned districts. 



Lessons from Philadelphia: A Leading Education 
Fund Helps Shape School Improvement 



Urban nonprofit education funds — Philadelphia, Boston, and Chattanooga, to name just a few — have been leaders and partners 
in data-driven educational improvement efforts. Several have taken their locally developed knowledge to the national level. 

The Philadelphia Education Fund (the Ed Fund) has worked to advance student outcomes in Philadelphia for the last 25 years. 

A research endeavor is fundamental to its mission, informing and supporting actions, and contributing to building the local 
knowledge, skills, and relationships necessary to work in partnership with the community. In 2004, researchers from Johns 
Hopkins and the Ed Fund and with the support of the William Penn Foundation, partnered to identify early reliable signals of 
students falling off the path to graduation. The seminal research led to middle school indicators that were disseminated in 
peer-reviewed papers, public forums, and became the basis for middle grades work in many other cities. 

The Power of Partnerships 

Knowing the relevant indicators, the Ed Fund partnered with the School District of Philadelphia to build the indicators into 
the school district’s student information system, enabling teachers, students, and parents, to access middle grade early warning 
indicator data.The Ed Fund and Johns Hopkins researchers also began a two-year pilot with a high poverty middle school to 
begin to learn how early warning indicators could be used to support more effective interventions. During this time, early 
versions of additional useful tools were developed. For example, a grid with attendance, behavior, and course performance on 
one axis and whole-school, targeted, and intensive interventions on the other axis helped schools take stock of their existing 
interventions and identified areas where they were lacking.This initial work also demonstrated that teachers would buy into 
and could quickly see the utility of early warning indicators as long as preliminary efforts were undertaken to explain how and 
why the indicators worked and attention was paid to building a sense of mission among the staff. It was also important to put 
processes into place to provide staff ready access to the data and time to analyze and discuss it. However, a key finding from this 
initial pilot was that the number of students identified with early warning indicators could become overwhelming. In high-needs 
schools with hundreds of students demonstrating offtrack indicators, there were not enough adults to meet all the needs of the 
students. 

These findings led the Ed Fund to partner with City Year of Greater Philadelphia and Communities in Schools of Philadelphia, as 
well as adopt the Johns Hopkins University Talent Development Whole School reform model (which in earlier years the Ed Fund 
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had helped support in Philadelphia). Ultimately, three north Philadelphia schools field-tested a whole-school intervention, which 
combined an EWS with a second shift of adults, integrated into the school fabric through an on-site facilitator.These efforts led 
to Diplomas Now (please see Diplomas Now case study for more information). 

Success Prevailed 

In the three pilot schools, by 2009-20 1 0, 56 percent fewer of the students were off-track in attendance, 53 percent fewer 
were off track in behavior, 82 percent fewer were off track in math, and 78 percent fewer were off track in literacy. After 
the U.S. Department of Education’s i3 award of a $30 million validation grant to the Diplomas Now model, based on the 
recommendations of a national review panel of other experts, the Ed Fund’s role expanded. It now serves as national technical- 
assistance consultant to Diplomas Now effort.The Ed Fund continues to analyze Diplomas Now data, together with Johns 
Hopkins, City Year, and Communities in Schools.The Ed Fund’s efforts provide an example of the role external partners can 
lay in complementing district resources, particularly when districts and schools recognize the need for data but are too under- 
resourced to analyze it. A current focus is on the effectiveness of interventions, the applicability of different interventions to 
different situations, and the reasons underlying effectiveness of particular interventions in schools. What policy and practice 
factors influence intervention outcomes positively and less positively? When districts and partners deepen understanding of 
these factors, schools will be enabled and empowered to address the dropout challenge even more effectively than current 
models allow. 



Professional Development, Coaching, 
and Networking 

Almost all EWS implementers identify professional development, 
coaching, and networking as a top strategy to guide hands-on 
use. In Louisiana, interviews with education leaders indicated that, 
with hindsight, deploying a group of well-trained staff into the 
field would have helped operationalize the seven-year old system 
more rapidly and effectively. In response, the State Department of 
Education, through the Division of College and Career Readiness, 
plans to commit 1 6 graduation coordinators statewide to improve 
graduation rates. Coordinators will spend 75 percent of their 
time coaching school personnel to effectively diagnose need and 
assign and track efficiency of interventions. Every school has been 
assigned a coordinator as a condition of the accountability system. 
The coordinators will target their work to assist high schools in 
the “red and yellow” alert categories. State and regional summits 
and conferences will enable educators to share best practices 
and what they have learned in the course of undertaking school 
improvement. 



In Alabama, the State Department of Education recently made 
the Alabama Graduation Tracking System, (AGTS) available to 
all its schools at no cost, to address the ABCs of increasing the 
graduation rate and reducing the number of students leaving 
K-12 schools. The process is systematic, timely and directive, 
and involves six steps for full implementation. To spur usage, 
Alabama has tied the AGTS process into the state-level system 
of professional learning credit for educators. Teams are currently 
being trained statewide in the implementation and use of the tool 
through regional workshops; participants practice downloading 
their own district or school’s data, and develop a plan to get their 
district or school organized in using AGTS. Additional workshops 
will be held for school registrars to practice protocols for coding, 
inputting, and uploading. Once back in the schoolhouse, the AGTS 
team is expected to work with colleagues to set local parameters 
and triggers that reflect local experience, and to reconsider 
policies.Triggers relate to attendance, discipline, grading, course- 
performance, how these factors and related policies influence 
graduation positively or negatively, and what refinements could be 
made to keep more students engaged in learning. Additionally, 25 
state-trained and supported coaches 
for the “high need” schools identified 
by the accountability system will add 
early warning system coaching to 
their “to do” list. Other districts and 



> Almost all EWS implementers identify professional 

development , coaching, and networking as a top strategy 
to guide hands-on use. 
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schools will largely rely on nearly 250 local personnel who have, 
at least once in the last four years, attended state workshops on 
coaching for graduation improvement. 



Meeting the Scale and Scope of the 
Identified Need 

The majority of states and districts that are at the leading edge of 
EWS are still in the early development and initial implementation 
stages. One key, yet unanswered question emerged from those 
that have been at it the longest: when early warning indicator data 
reveals large numbers and high concentrations of students in need 
of additional supports, how do you effectively respond? In the 
most challenged schools, hundreds of students can be off-track 



for graduation. Without adequate supports, a common reaction 
of front-line educators and student support personnel is to adjust 
down threshold levels to focus on a smaller, and presumably 
more manageable number of the 
highest needs students.This, while 
understandable, is not always the most 
productive strategy. Rather, students 
who are just at the indicator threshold 
or are moving in that direction are 
those who can most likely be put 
back on track to graduation with existing school personnel. 
Comprehensive efforts are necessary in these high-needs schools, 
combining whole school reforms, EWS, and a second shift of 
adults recruited from nonprofits and the community.These efforts 
should be tied together in an integrated school improvement and 
student support system that enables all students, no matter their 
numbers, to receive the supports they need to stay on track. (See 
pullout box on Diplomas Now for additional information on how 
to address these capacity challenges.) 



> Comprehensive efforts are necessary in these high-needs 
schools , combining whole-school reforms, EWS, and a 
second shift of adults recruited from nonprofits and the 
community. 



Diplomas Now: 

Whole-School Improvement with EWS 



Diplomas Now (DN) is one of the first whole-school improvement models to explicitly incorporate an EWS. It is designed to help 
the most challenged schools in America’s largest cities succeed in helping every student graduate ready for college and career.The 
initiative blends a system of early warning indicators paired with enhanced student supports and interventions into the larger context 
of comprehensive whole school reform. Diplomas Now was the only secondary turnaround group with national reach to win a $30 
million federal Investing in Innovation (i3) validation award (plus a $6 million matching grant from the PepsiCo Foundation). Partners 
are Talent Development Secondary Schools, Communities in Schools, and City Year. Current sites include middle and high schools 
in California, the District of Columbia, Florida, Illinois, Louisiana, Massachusetts, Michigan, New York, Pennsylvania, and Washington. 
Overall, the initiative will expand to 60 middle and high schools in 10 districts over the next five years. 75 



The model is simple but powerful: Diplomas Now “sets goals based on students’ attendance, behavior, and course performance. They 
develop a strategic plan, implement an early warning system to identify struggling students, and regularly review the data.Teachers 
and the Diplomas Now team craft individual student plans that include more math and English time and teacher teams with shared 
planning time. For the neediest students, Diplomas Now helps form support groups and connects them with community resources, 
such as counseling, health care, housing, food, and clothing.” 76 Translating the model into reality requires strong leadership from the 
principal, a DN site team supporting teams of teachers, early warning indicator data, and often, local community agencies. The DN 
approach is infectious and successful. It not only helps schools implement an integrated set of evidence-based school improvements, 
it also provides people to help put them in place and respond to what can be an overwhelming number of students in need of 
targeted supports. 



The DN site team includes a school transformation facilitator from the Talent Development Secondary Schools program. This 
facilitator acts as a key aide to the principal, supports teacher teams, helps to operate the EWS, and analyzes data to deduce which 
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types of supports are needed and are being effective. The DN site team also typically includes a team of 10 to 12 City Year corps 
members, who are doing a year of national service supported by AmeriCorps. Each corps member is assigned to a homeroom 
of students and within it a focus list of students who have exhibited an off-track indicator. Corps members help these students 
throughout the school day, serving as a near-peer role model. Finally, the DN site team has a Communities In School site coordinator 
who leads and organizes the case managed supports required by students facing significant out of school challenges. 

Examples of DN in Action at the School Level 

* A 1000-student middle school in which 95 percent of the students are eligible for free- or reduced 
price lunch. The principal and DN administrator are passionate that their students succeed. Watchful and encouraging 
administrators at every intersection and the cafeteria keep students moving to class on time. Everyone learns the rules 
because every adult is on the same page. Teachers are focused on instruction; vertical teams met within departments to 
discuss student performance in the content areas, and persuaded the district to incorporate outcomes of teacher-made 
benchmark tests into EWS.The DN administrator keeps DN team meetings moving along smartly, with two-minute 
discussions of “watch list” students prepared in advance by the DN facilitator, from district- and school-provided data. 
Questions come from all directions. Is the student slipping or improving in attendance, behavior, and coursework? What 
interventions did the math teacher try? Does anyone know why Student B missed school for several days, and who spoke 
with that student's parents? DN team members — teachers, one counselor, the Communities In Schools representative, and 
six City Year mentors — bring their professional understandings to the table and quickly reach agreement about next steps 
to try in the next two weeks. The school recently made Adequate Yearly Progress; reading scores are rising, and math 
scores, though not yet at the desired level, are also doing so. 

* A small K-8 urban school in which there are 150 students in sixth through eighth grade. More than 80 
percent of the students are “on track” in attendance, but overall, math, literacy, and behavior are a challenge. Close to 40 
percent of the students “off track” in the first two, and half are “off track” in behavior.The principal is deeply supportive 
of her staff and they return her support with respectful candor about difficult topics. The DN team considers charts of 
student progress on the different indicators, prepared within the school and posted in the DN team meeting room. The 
local education fund re-analyzes the data by marking periods to determine improvement in response to different types 
of interventions. Emerging data suggests that classroom-based problem solving is most effective in math classes while 
individualized tutoring works best in English language arts. 

* A 400-student high school serving students who are 97 percent African American and 78 percent free- or 
reduced-price lunch eligible. In the previous few years, enrollment has declined precipitously, as have state assessment 
scores, and a new principal and administrative team are working to turn the school around using DN as the driver.The 
Talent Development/DN model of a ninth grade academy was implemented as was a summer “boot camp” for rising ninth 
graders; students take math and literacy proficiency tests as well as state assessments to augment teachers' understanding 
of their needs, the DN facilitator prepared a sheet for each ninth grader showing progress on course-passing required for 
promotion in this state, and active City Year mentors are evident throughout the ninth grade classrooms. Each member 
of the City Year DN team assumes responsibility for a set of students from the watch list, although in the end, they work 
with all — as a student reported, “I imagine in the end they have touched the lives of most of these kids.” Working with 
the DN Communities in Schools site coordinator, the school’s parent coordinator (supported by Title I funds) created an 
ambitious involvement and literacy program that fosters communication and competition among parents, and has brought 
more than I 30 parents to school once a month over the school year for a reading program. 

Overall, DN has been able to reduce the number of students showing off-track in attendance and behavior by half and the percent of 
students failing courses by three-fourths, with some variation by site, and implementation conditions. 
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Louisiana:Accountability Pulls Up 
a State Education System 



One in three children in the Bayou state live in poverty — one of the highest rates in the nation. To complicate the struggle, six years 
ago, the state was confronted with one of the country’s worst disasters in recent years. Despite these obstacles, policymakers and 
educators in Louisiana have spent more than a decade crafting a data-based educational accountability system as a framework for 
helping schools improve student outcomes. Initial state feedback of data to districts began in the late 1 990’s. Refinements have 
continued, first with growth of the state longitudinal data system — one of I I deemed by the Data Quality Campaign (2010) to have 
the 1 0 essential elements for functionality — and then with development of the Louisiana statewide Dropout Early Warning System, or 
DEWS, arguably the first among the state EWS. 

Recognizing early on the leveraging power of attendance, graduation rates, and diplomas, Louisiana factored K-8 grade attendance, 
high school graduation rates and type of high school diplomas awarded into the performance score used to reward and sanction 
schools, beginning in 2003. When the Graduation Index counts for 30 percent of a school’s score, and schools are incentivized with 
additional points for the type of diplomas awarded, they begin to pay attention. Gradually, the state cohort graduation rate has risen: 
for 2009-2010 it is 67.2 percent, up nearly six percentage points over the last 10 years. On-lookers attribute gains to cleaning up the 
data, using data, policy changes, and dedicated education professionals fully committed to improving student outcomes. 

Data In. Data Out. 

DEWS began in 2004, inspired in part by the success of a high school principal and his leadership team in using data to identify 
students at risk of failing. As with most EWS, DEWS has continued to develop since its inception, more recently in partnership with 
the EDGEAR/JPAMs student information system collaborative at Louisiana State University. Technical aspects of the DEWS system 
reflect some used in other EWS along with some innovations. 

• After extensive analysis, 200 possible factors influencing graduation were reduced to the three most important indicators — 
attendance, behavior, and course passing. 

• Time was factored into indicators and triggers. Special attention was paid to the first 30 days of schools, as students with 
significant absences during this period had severely impacted grades. Students whose GPA dropped .5 were flagged, as were 
students with Ds. 

• Other thresholds were set based on percentages rather than absolute cut points. 
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• Flexibility is allowed in schools and districts. They have the option of raising the bar higher for thresholds, just as they have the 
option of using or not using the system. 

• Various individuals carry out data input, depending on school and district organization. This includes classroom teachers, 
attendance clerks, and disciplinarians. 

• DEWS was first piloted in 20 rural and urban schools, with input gathered about effectiveness. Later, state grants to 
approximately 1 75 middle and high schools fostered review of data, early and often, targeted to support students. 

• Initially, DEWS provided superintendents, principals, and leadership teams with lists of students slipping into trouble on the 
first and fifteenth of every month. Others in the school received role-specific reports and counselors now both receive a daily 
report of students who have moved into “at risk status” and are able to track the effectiveness of interventions. Reports are 
delivered automatically over email, reducing the burden on educators. Schools continue to use student information systems of 
their choice, although approximately 75 percent go through the LSU EDGEAR system. Those which do not can receive real-time 
data and run their own reports. 

What They Learned and Where They Went 

With hindsight, these educators know that making the system easy to use is key, and that deploying a cadre of people in the field 
would have helped to rapidly and effectively operationalize use. Now: 

• Schools will receive a Graduation Support Profile for each incoming ninth grader, covering age, and eighth grade 
mathematics and English language arts scores on the LEAP (Louisiana Educational Assessment Program test), discipline, 
mobility, and attendance. 

• The State Department of Education, through the Division of College and Career Readiness, will commit 16 graduation 
coordinators statewide to improving graduation rates. Coordinators will spend 75 percent of their time coaching school 
personnel to effectively diagnose need and assign and track efficacy of interventions. Every school has been assigned a 
coordinator and a condition based on the accountability system; the coordinators will especially assist high schools in the “red 
and yellow alert” categories, vertical teams of middle and high school personnel, and middle schools when these are recipients 
of federal improvement grants. 

• State and regional summits and conferences will enable educators to share best practices and what they have learned in the 
course of undertaking school improvement. 
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Now more than ever, education systems must harness the power 
of data to effectively serve students and society. Our investigation 
shows that EWS are at the cutting edge of high-impact education 
improvement initiatives. Their synthesized data, when appropriately 
leveraged, can not only help sustain the investments made in early 
education, but also accelerate gains in academic performance for 
students at all levels. EWS based on the pairing of research-based 
indicators and evidence-based interventions have the potential 
to be a game-changer for our nation’s struggling schools. Even 
schools with limited resources can implement EWS and use these 
systems to help all students graduate from high school prepared 
for postsecondary success. 



EMERGING BEST PRACTICES: FOUR KEY 
QUESTIONS FOR SUCCESSFUL EWS 
PLANNING AND IMPLEMENTATION 

A number of guidebooks and tools supporting the use of EWS 
are freely available over the web or for purchase (see Appendix 2 
for additional information). A resource gap still exists. In order to 
help close this gap, four key planning and implementation questions 
and subsequent recommendations are outlined below for parents, 
educators, administrators, district leaders, state Department of 
Education officials, and policymakers to consider. 



Stakeholders from all sectors, including parents, 
educators, administrators, legislators, business 
leaders, and funders, can apply the lessons learned 
from the early adopters of EWS to the content 
of their work.This section focuses on emerging 
best practices, needed research, and policy 
recommendations. 



> The Four Key Questions to Guide EWS 
Implementation Are: 

• What Types of Data Will Be Recorded? 

• How Will Data Be Recorded? 

• How Will Recorded Data Be Used? 

• Who Should Be Consulted in the Planning and 
Implementation Phases of EWS and 
What Are Their Ro les? 
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Key Question I : 

What Types of Data Will Be Recorded? 

Put the student first. 

Data and students must both be treated as important components 
of an EWS. Data help to identify students and craft interventions, 
but the success of the student is the ultimate goal. So, while it is 
important to rapidly identify students who are falling off track, it is 
also important to identify and build on their strengths. 

Use empirically created indicators. 

Powerful indicators can be identified based on the analysis of 
longitudinal data that tracks individual student progress over time. 
In essence, examining the behaviors associated with dropout 
among previous cohorts of students will help to determine 
which indicators ought to be included in the EWS and which 
interventions ought to be paired with the selected indicators. 

Be selective in the set of indicators. 

As technological systems have advanced, and the capacity to 
integrate data from multiple sources has improved, it is possible to 
generate a dizzying variety of data so that people are information 
rich and knowledge poor.A few key variables are easier for 
teachers to monitor than a large set of predictors. They can 
reduce the burden of extra work and help inform instruction. K- 1 2 
analyses have demonstrated that although the underlying issues 
that produce the poor grades or weak attendance may be complex 
and may vary from student to student, there are a small number 
of behaviors that alert educators to students potentially falling off 
track. By extension, a good indicator system also identifies variables 
that are not the strongest predictors of eventual dropout, and does 
not depend on them. 

Focus on “high yield” indicators to capture the majority 
of students who eventually become dropouts, rather 
than identify small minorities of students with 
uncommon manifestations. 

A good indicator system avoids the “one percent problem” — 
indicators that are highly predictive but that only identify a small 
percentage of dropouts. The most highly predictive high yield 
predictors are: 20 days of absence (one month of school), failing 
two or more courses, and sustained mild misbehavior. Additional 
research, time markers, and local nuance strengthen the predictive 
accuracy of these indicators. 



Begin to respond to student behavior well before triggers 
for more intensive interventions are reached. 

Twenty missed days may signify that the danger zone for 
attendance has been crossed into, but students should never be 
allowed to reach this level of absence. Successful schools keep a 
watchful eye on all students in the first 20 days of school and begin 
to intervene as soon as several absences not associated with an 
illness occur.The same vigilance is required regarding academic 
performance and behavior. 

Determine policies for sharing information with and 
collecting data from sources beyond the schoolhouse 
to enhance knowledge of students and to target 
interventions most effectively. 

There are many sources of helpful information in youth-serving 
community agencies external to schools or the district. For 
example, local health and welfare agencies, nonprofits that offer 
extended learning time, mentoring and tutoring, and the juvenile 
justice system may have data and information that could help to 
increase knowledge of students and to determine the interventions 
that would best meet student needs. Determine how this 
contextual information can be used to guide students back onto 
the graduation path. This data can also be used at the school and 
district levels to adjust resource allocations and to share resources 
across agencies. Schools with very high populations of agency- 
involved youth that are demonstrating off-track indicators should 
be supplied with the resources required to address the unique 
challenges these young people face. 

Recognize there will be state-by-state and district-by- 
district variability in data availability, data system capacity, 
and support for interventions. 

Different states have different philosophies and policies about what 
is appropriate to collect from districts and schools, and how often 
the information should be collected. States and districts also vary 
in terms of financial, human, and technological capital that can be 
applied to building and sustaining EWS. As a result, in some states 
it will be possible to have strong state supported EWS efforts, 
while in others, districts, community organizations, nonprofits, 
and/or schools will have to take primary responsibility for 
implementing these systems. Federal and state efforts to incentivize 
the spread of EWS should allow for their growth whether 
system implementation is led by the state, district, community 
organizations, nonprofits, or corporate partners. 
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Key Question 2: 

How Will EWS Data Be Recorded? 

Record data from the simplest and most direct 
source possible. 

EWS can be implemented as early as the later elementary school 
years and should cover key transitions (i.e. sixth and ninth grade). 
The running record found in a teacher’s grade book, whether on 
paper or electronically, is the most important data for EWS. Grade 
books show, by period in middle and high school, who was present, 
who came late, who turned in work, who received what grade, and 
how they behaved in class. Records in the registrar's or data clerk’s 
offices, and those of the counselor’s, assistant principal’s, or dean’s 
augment teachers’ records. Data collection does not need to be 
overcomplicated, but it does need to be accurate. As EWS become 
refined, data collection may be increasingly automated. 

Ensure data are entered by appropriately trained 
personnel and according to well-designed protocols. 

The quality and utility of a data system depends on the accuracy of 
the data stored within. Data must be consistently coded and coding 
protocols must be followed daily. Adequate staffing of data entry 
initiatives, with a mindfulness of current staff caseloads, is essential 
for success. 

Key Question 3: 

How Will EWS Data Be Used? 

Explore issues of privacy. 

Ensure that children’s privacy is protected while also leveraging 
data to most effectively promote their success. Ensure adult 
stakeholders are comfortable using data with respect to privacy 
regulations by broadly communicating how they are allowed to 
use what types of data. It will be important at the federal level to 
provide a range of examples of how EWS data can be effectively 
used within existing privacy laws, as some districts are reluctant 
to proceed for fear of violating laws and regulations they do not 
fully understand. To address these issues, EWS can be structured 
to provide role-specific access tailored to each adult’s specific 
interactions with students. 



Use the advantages of technology to reduce information 
into easy-to-understand data presentations. 

Convert student-level data reported by teachers, counselors, and 
administrators into simplified reports requested by these users, 
showing patterns and progress, with dashboards or stoplights and 
clear flags. Transparency and usability should be the goals for 
these reports. 

Teach people how to understand and use data. 

Provide training and professional development to help educators 
and administrators learn how to effectively leverage the power 
of data and to collaboratively work to keep students on track to 
graduate from high school prepared for college and a career. Invest 
in mission building efforts so teachers and other school staff can 
learn how attendance, behavior, and course performance not only 
impact student’s academic success within their classrooms, but also 
shape their educational trajectory for years to come. Additional 
support can be provided and learning communities fostered via 
webinars, videos, and conference calls. 

Provide follow-up coaching for data use. 

Initial training is important. Experience has shown, however, that 
not all individuals are equally comfortable using data. When coaches 
are available to support their colleagues on a frequent basis, data 
are used most effectively.These coaches can be drawn from the 
ranks of master teachers, other instructional leaders, and school 
personnel who appreciate numbers and patterns Weekly meetings 
have proven most effective. 

Compose a “support list” of students and revise it every 
few weeks based on the indicators. 

Track student data over time. Some students will improve. Others 
will slide. Data analysis needs to be an ongoing process. 

Act on the data shown in the “support list” and 
associated reports. 

Get educators to analyze and discuss individual student-level 
data frequently, asking the question, “What next steps can we 
as adults take to better understand and then help change this 
student’s situation?” 
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Build EWS meeting time into the school schedule and 
guard this meeting time zealously against intrusions. 

As educators begin to see results, they will no longer perceive 
EWS as burdensome time, but as valuable opportunity for collegial 
discourse that informs their understanding of their students and 
their instruction. When students succeed, teachers succeed. That 
positive reward drives many educators. 

Make decisions about actions and interventions as a team. 

Hold student-level discussions among teams of teachers and 
other adults who interact with the same students whenever 
possible. Having a discussion leader and a protocol to facilitate the 
conversation and keep it on track is extremely useful. Two-minute 
discussions every two weeks per “support list” student are in most 
cases sufficient, manageable, and highly productive. 

Look for and act upon patterns that emerge from the data. 

In many cases, the number of students demonstrating off-track 
indicators will be greater than the school has the capacity to 
respond to with individually tailored interventions. It will be 
important to implement school-wide prevention activities when 
large numbers of students are chronically absent, getting in trouble, 
or failing courses. If students who are signaling that they are falling 
off the graduation path are heavily concentrated in a sub-set or a 
few classrooms, the most judicious intervention might be to work 
directly with the teachers in those classrooms, providing them 
additional support and professional development. 



Know your school and students when 
choosing interventions. 

A school with 600 ninth graders, 40 percent of whom are not 
coming to school and are failing English or math, and a school 
with 300 ninth graders, 10 percent of whom are failing two or 
more courses, have quite different situations.The solutions are 
equally variable. In the first case, principal and leadership team 
vision coupled with school-wide prevention strategies and external 
support from community agencies and nonprofits may be essential 
to improving outcomes for the 240 struggling students. In the 
second case, an analysis of instructional practice and curriculum 
supports may go a long way towards improving the achievement of 
the 30 struggling students. 

Practice intervention discipline. 

A human reaction is to focus time and effort on the neediest 
students. Often, however, a professional case manager best serves 
these students.The students who have just fallen offtrack, or are 
heading in that direction, can be most easily guided back onto the 
graduation path through the collective efforts of their teachers or 
nonprofit partners. Thus it is important to reserve time and energy 
and resources for these students, while making sure that the 
highest need students get the more intensive supports they need. 
The latter can often be helped by augmenting social workers and 
counselors with individuals from wrap-around support agencies. 
Especially in schools with large volumes of off-track students, it is 
important to form partnerships with nonprofits that can provide 
the human capital at the scale and intensity required to support 
moderate intensity or “Tier 2 ” interventions, like attendance 
monitoring, positive behavior reinforcement, and help with 
assignment completion. 

Engage stakeholders from various backgrounds in 
data usage. 

Parents, teachers, administrators, nonprofit partners, and even 
students should be incorporated into EWS usage. With technology, 
various levels of access can be established so that, for example, 
parents can access only their own child’s portfolio or nonprofit 
partners can communicate with teachers and parents about the 
students they serve. 
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Track outcomes of interventions frequently. 

The purpose of the team meetings to discuss students on the 
“support list” shifts over the course of the year.Throughout the 
process, keep the discussions short, frequent, and focused. In the 
beginning, team meetings help adults decide which interventions 
to apply. As the year progresses, team members will want to 
analyze whether the interventions are working. For meetings 
among community agencies and school personnel, outcomes of 
interventions may be assessed over longer periods, bi-weekly, 
monthly, and quarterly for formative purposes, and/or by semester 
and annually for summative and reporting purposes. 

Keep the goal in mind: the primary objective of EWS 
should be to keep students on track to graduate high 
school, college and be career ready. 

The highest impact EWS focus on two sets of students: students 
who ended the prior year with an off-track indicator, and students 
who did not exhibit an off-track indicator in the prior year, but 
in the current year are sliding towards one. Increasingly, districts 
are providing schools lists of incoming students who may be 
falling off track. Typically, these are sixth and ninth graders, who in 
the fifth grade or eighth grade had poor attendance, behavioral 
troubles, or failed courses. The goal of providing this information 
is both to ensure offtrack students start the new year on track 
and stay there, and to provide students who are sliding off the 
tracks targeted supports to keep them from falling offtrack. Finally, 
schools with large numbers of students can develop off-track 
indicators to identify the need early for school-wide preventive 
efforts like attendance campaigns, positive behavior modeling and 
recognition systems, and polices and supports to enable rapid 
academic recovery. 

Key Question 4: 

Who Should Be Consulted in the Planning 
and Implementation Phases of EWS and 
What Are Their Roles? 

Provide local leadership for early warning and 
indicator systems. 

Every early warning indicator and intervention system needs a 
“champion” who will advocate for it constantly at the school, 
district, or higher level. Champions come from varied sectors — 
superintendents, associate superintendents, directors of research 
and assessment or technology, principals, teachers, community 



leaders, nonprofit directors, and business personnel. But 
“champions” can’t succeed alone. At whatever level it is relevant, an 
advisory team offers important support and contributes knowledge 
from different perspectives within the community that cares about 
student outcomes. This team may be composed of an administrator, 
an instructional representative, a counseling or social work 
representative, and a data-savvy person such as a science teacher 
who loves numbers and patterns. 

Have a development and implementation plan 
and timeline. 

The champion and advisory team’s first step is to come up with a 
development plan, followed by an implementation plan, recognizing 
that timelines may shift as opportunities and challenges arise. 

A development plan will include costs, which are highly variable 
because they are determined by pre-existing hard and operating 
system platforms, data systems, the potential for the data system to 
be modified, etc. 

Listen to the end-users and find out what they want before 
going too far. 

Designers of early warning indicator and intervention systems 
should convene focus groups of end-users-administrators, teachers, 
counselors, social workers and others-to find out the format in 
which short data reports and longer analyses are most useful. 

Build up from a pilot. 

Start with a manageable group of implementers. Depending on 
capacity constraints, this could be a cohort of students, a single 
grade level, a whole school, or even a district. Take the lessons 
learned from this pilot and apply them when scaling. 

Encourage schools to share outcomes of interventions and 
learning about early warning indicators with schools and 
districts with similar demographics and constraints — as 
well as those that are dissimilar. 

Solutions can be found in many places. Collaboration leads to 
better results and new ideas for changing cultures. When teachers, 
community partners, and administrators receive and discuss data 
and interventions in ways that are appropriate to their roles, early 
warning systems and data analysis become part of the culture of 
the school and community and lead to more informed decision- 
making and actions. 
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Integrate EWS into instructional improvement efforts and 
other student support services. 

High performing EWS link efforts to keep students on the 
graduation path with school-wide efforts to improve instruction. 
The link with instruction also helps with teacher buy-in, as they 
see how making sure students attend regularly and fully participate 
creates classroom conditions which enable more effective 
instruction. EWS can also be integrated with growing efforts to 
infuse data-driven decision making into instructional improvement 
efforts by having teams of teachers analyze student work and 
benchmark assessments to improve classroom practice, curricular 
design, and pedagogy. As typically there is only one teacher team 
meeting in the schedule, it is essential that EWS and data-driven 



instructional improvement efforts be thoughtfully linked. Finally, 
many schools have Student Support Teams (a team of composed of 
an administrator, teachers, counselors, and other student support 
personnel) who meet regularly to review students who are 
struggling and design interventions.These teams can function as a 
natural leadership group to introduce, support, and leverage EWS. 

Engage stakeholders beyond the school system. 

EWS are only effective when paired with research-based 
interventions. Teachers and counselors are often well equipped 
to provide these supports but others may be bound by capacity 
constraints. Educators can contact local and national nonprofits to 
help meet the need of the students and families served. 









What are the financial costs to an EWS? 



Cost projections are part of the early planning stages for an EWS. Resources, funding, staff, or external consultant time will need to 
be devoted both to identifying the indicators and thresholds to be used by that locale or district, and the degree of flexibility that, 
depending on the organizational level, is permitted for schools and districts. The functionality of the student information system and 
the existence or potential for creating or accessing a data warehouse must be evaluated. Methods of data synthesis and redistribution 
need examination and decision. Will distribution be simple, through an excel spreadsheet, will it occur through a complex data- 
aggregator that automatically produces pre-selected reports judged to be most useful, with different privacy settings for different 
users, or will there be flexibility in querying? 

Costs associated with these investigations and discussions vary depending on circumstance, but are not huge, as they primarily 
involve staff time or consulting contracts of limited duration. Modest and slightly longer lasting costs are associated with proving 
implementation supports for the adults using EWS including initial training, follow-up coaching, and technical assistance. Larger costs 
are involved in providing student interventions at the scale, duration, and intensity required. Here, however, existing revenue streams 
like Title I, School Improvement Grants, and state compensatory funding can be tapped and more effectively directed. 
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NEEDED RESEARCH TO ENHANCETHE 
NATIONAL KNOWLEDGE BASE 

Much is known about early warning indicators, but much less 
is known about the complex set of conditions and policies that 
underlie the most effective implementation and outcomes. Three 
actions are recommended to advance the knowledge in the field: 

Create federal, state, and district-level workgroups to 
study and guide dropout prevention and graduation 
improvement efforts with a specific focus on policies 
that may contribute to or distract from increasing 
high school and college and career readiness rates, 
on a state-by-state basis. 

Substantial differences exist surrounding the conditions of 
education in the different states. The list of policies which influence 
graduation is long: graduation requirements, course and credit 
requirements, pupil progression policies (promotion policies grade 
to grade), retention policies, diploma types, opportunities for 
exemptions, policies related to double jeopardy for absence and 
suspension, or absence and grades, opportunities for suspended 
and/or expelled students to re-enter educational settings and 
resume their education, the age at which states are no longer 
required to offer free high school education to their students. 

These policies are worthy of reexamination in light of the renewed 
U.S. commitment to graduating greater percentages of students 
who are better prepared. 

Conduct state and district-level surveys. 

Determine the extent to which early warning indicator and 
intervention systems are being implemented in districts and 
schools, and the characteristics of the districts which have 
implemented these compared with the characteristics of those 
which have not. 

Conduct design studies, multi-site implementation 
analyses, and random control trials to determine the 
characteristics of the most effective early warning 
indicator and intervention systems. 

Despite the fact that early warning indicator and intervention 
systems are based on sound research and appeal to common sense, 
a well-controlled study that examines impact has not yet been 
conducted. As part of the scale up of EWS, sound research on their 
effectiveness, under different conditions, should be conducted as 
soon as possible. 



POLICY RECOMMENDATIONS 
PERTINENTTO THE CIVIC MA RSHALL 
PLAN AND INCR EASING THE 
GRADUATION RATE 

The work of the Alliance for Excellent Education projects just how 
much the dropout crisis costs each state in terms of lost revenues 
and new costs. 77 Families, teachers, counselors, administrators, and 
nonprofits are hard at work to help children achieve. Policymakers 
can significantly advance this ground-level work by enacting high- 
impact levers at the district, state, and federal levels. 

The Civic Marshall Plan (CMP), part of the larger Grad Nation 
campaign, strives to ensure that 90 percent of the Class of 
2020 will graduate from high school prepared for college and 
a career. EWS are identified as a key lever in the CMP.The 
recommendations that follow are organized around the Plan’s 
three key recommendation areas. For a complete list of CMP 
benchmarks, please visit 
www.civicenterpi-ises.net/reports.php 

I ) Take Action within Low Graduation 
Rate Communities 

Fifty percent of America’s dropouts come from just 15 percent of 
schools. These “dropout factories” can be targeted for intervention, 
dramatically accelerating the high school graduation rate. EWS can 
be strategically leveraged in these schools to target interventions 
to meet the needs of students at risk of dropping out through the 
following actions: 

At the Local Level 

Harness the Power of Nonprofits. 

Nonprofit organizations have much to offer schools as bases for 
research, as sources of experienced planners, and as providers 
of interventions when schools require additional capacity and 
resources. For example, the successful pairing of EWS with 
nonprofit collaborators in Baltimore, Boston, Chattanooga, Chicago, 
Colorado, and Jacksonville, to name just a few, offers insights as to 
what can be achieved through partnerships. Proven interventions 
such as Big Brothers and Big Sisters, Boys and Girls Club, City Year, 
Citizen Schools, Communities In Schools, Diplomas Now, and EWS 
efforts launched by the United Way, have proven their worth with 
innovative, highly-targeted and high-impact efforts that produce 
results around the country.These collaborations are especially 
important for students who are involved with other agencies (e.g. 
juvenile justice and foster care), who may require specialized support. 
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Harness the Power of Corporations. 

The business community has much to contribute to the successful 
design and implementation of EWS. For example, the Knoxville 
Chamber’s work to adapt business acumen into the design 
and implementation of an educational technology system that 
addressed not only student progress but the cost-effectiveness of 
the district’s resource allocations and educational investment is a 
community-based example of corporate contributions to education 
redesign. Likewise, Nationwide Insurance in Columbus, Ohio 
helped their local district build its EWS system and then created a 
nonprofit to help spread it to other communities. These business 
investments should be celebrated and best practices should be 
shared. To learn more, please see “Education as a Data-Driven 
Enterprise:A Primer for Leaders in Business, Philanthropy, and 
Education” available at www.civicenterprises.net/reports.php 

.At the Local and State Level 
Link EWS to the ABCs. 

Research has consistently shown that Attendance, Behavior, and 
Course Performance are the strongest predictors of whether a 
student is on-track to graduate. Require the use of data-driven 
and research-based criteria to define each of these three dropout 
indicators. Utilize research-based cut points, including 20 absent 
days, two or more course failures, and sustained mild misbehavior, 
which are highly predictive measures of a student’s risk of dropping 
out.Tie these ABC indicators, and the research-based cut points, to 
EWS and high school completion efforts. 

At the District, State, and Federal Level 

Align graduation acceleration efforts with college and 
career readiness efforts through the Common Core. 

Our nation is facing not just a high school dropout crisis, but 
also an accompanying college completion crisis and labor market 
skills gap. School, district, and state initiatives related to dropout 
prevention, graduation acceleration, and college and career 
readiness should be aligned with EWS, including funding streams, 
policies, and regulating offices. Specifically: 

• At the School and District Level, provide technical 
assistance to educators so that they understand the use and 
efficacy of Common Core Standards. In addition, benchmark 
assessments linked to the Common Core, as well as emerging 
college readiness indicators, need to be integrated with EWS 
to create a unified and streamlined approach to using data 
to improve student outcomes. This will help to ensure that 
educators can view and act upon these data sets in unison so 
that the strongest outcomes will be achieved. 



• At the State Level, adopt Common Core Standards and 
create dedicated offices or inter-departmental efforts in state 
Departments of Education and in larger districts to guide 
school improvement efforts centered on dropout prevention 
and graduation improvement, connected to college and career 
readiness. Ensure the work of these offices is integrated with 
school improvement, reform, and turnaround work. 

• At the Federal Level, Congress should encourage 
states to adopt college and career ready standards through 
the reauthorization of the Elementary and Secondary 
Education Act. 

Drive attention to the middle years. Build the 
infrastructure to support the development of EWS 
as all grade levels. 

Habits are formed, and knowledge and skills built, prekindergarten 
through first grade as well as later.Additional research is needed 
to identify which indicators and interventions are most effective in 
the early grades. The research base is clear that the early years of 
adolescence are a critical time when students are launched on a 
path towards high school graduation and college readiness or begin 
to disengage from school and start the downward spiral that often 
ends in dropping out. 



Ensure that federal funds, especially Title I dollars, are 
targeted to the highest need schools. 

This can be accomplished by providing additional financial support 
to schools in which 75 percent or less of the senior class graduates 
from secondary school. It is particularly important that funds 
disbursed to support School Improvement (authorized by Title I of 
ESEA) and the High School Graduation Initiative (as authorized by 
Title I Part H of ESEA) be disbursed to schools in which 75 percent 
or less of the senior class graduates on time from secondary school. 

* Target funds and attention to districts and schools 
most in need, particularly in rural districts and 
schools. The technology and the human capital required 
to implement an EWS may be lacking in under staffed rural 
districts where there is a low tax base and/or adults are 
already responsible for filling a variety of roles. EWS can make 
struggling districts’ jobs easier, enabling them to target work 
earlier, when lighter, less expensive, interventions are effective. 
State-sponsored EWS solutions and interventions will go a long 
way toward alleviating rural graduation challenges. Rural schools 
looking for resources can also turn to Rural Development 
state offices for support as well as to The Rural Education 



At the Federal Level 
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Achievement Program (REAP) Rural and Low-Income School 
Grant Program for funds. REAP provides grants to build and 
utilize educational technology, including EWS. 

2) Build and Enable State and District 
Capacity to Improve Graduation and 
College Readiness Rates 

At the District and State Level 

Invigorate existing policies that encourage graduation and 
implement best practices. 

Many states have “achieve to drive” policies and “pass and play” 
requirements for athletes. Others have laws linking dropping 
out to driving license suspension. In the past few years, with the 
encouragement of the Civic Marshall Plan Leadership Council, 20 
additional states have increased their compulsory school age law 
to 18, but this is not enough. 78 Ensure these best practices are fully 
implemented, and replicated in other states and districts. Examine 
policies that may discourage graduation, such as double jeopardy 
policies that react to one negative indicator (e.g. poor attendance) 
by enacting another (e.g. suspension), or polices which mandate 
course failure if students miss a certain number of days. 



Convene a summit of early adopters of EWS. 

Include school, district, and state-level representatives, as well 
as community-based organizations and legislators, to share best 
practices and accelerate effective EWS usage. 

Utilize funding streams that can support efforts on 
the ground. 

State Departments of Education can leverage federal dollars to 
support the development and implementation of EWS. Federal 
funding sources that states and districts may consider are:Title 
I Section 1003(g) of Elementary and Secondary Education Act 
(ESEA), Title I Part A of ESEA, and Title I Section 613 of The 
Individuals with Disabilities Education Act (IDEA). 

Collect information on the status and challenges of 
EWS implementation. 

At the state level, through the addition of EWS-focused questions 
in the annual Data Quality Campaign (DQC) survey.The DQC 
conducts an annual state analysis of state data systems that 
measures progress towards building and implementing the 10 
Essential Elements of Statewide Longitudinal Data Systems and 
the 10 State Actions to Ensure Effective Data Use; EWS-focused 
questions could be added into these annual surveys. Carry out 
similar efforts with districts through the National Center for 
Education Statistics (NCES) or other organizations. 
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At the State and Federal Level 

Ensure that investments in developing the data systems 
needed for EWS are consistently paired with efforts to 
build intervention systems and policy reviews, particularly 
at the state and federal level. 

Early warning data has enormous potential to improve educational 
outcomes that will only be realized if EWS efforts go beyond 
student identification. Early warning data enables smarter, more 
effective, timely, and cost-efficient interventions, but these benefits 
are only realized when schools have the capacity to meet the 
identified student needs. 

At the Federal Level 

Require EWS for federal education funds. 

Incentivize states, districts, and schools around graduation rate 
improvement. Require states, districts, and schools with less 
than 75 percent graduation rates, and which receive federal or 
state school improvement grants of any kind (including Title 
I , School Improvement Grants, Perkins funds, compensatory 
funding, etc.) to implement an early warning indicator and 
intervention system based on longitudinal data analysis. States, 
districts, and schools with a 10 percentage point gap between 
overall and sub-group graduation rates should also be required 
to implement an early warning indicator and intervention 
system in order to receive funds. 

* Amend federal regulations involving the 

Supplemental Educational Services (SES) section 
of Title I to encourage and enable EWS. This change 
could happen either through re-authorization of ESEA or the 
emerging waiver process. SES was originally designed in 2001 
to provide tutoring supports primarily to elementary students. 
Since then, the EWS research has shown the importance 
of supporting students later in their academic career and 
keeping students on track to graduate from school college- 
and career-ready.The supports required include tutoring, but 
also involve attendance monitoring, behavioral supports, and 
assistance with assignment completion, among others. The SES 
provision ofTitle I needs to be re-fashioned so that schools 
and districts can strategically partner with proven national and 
community nonprofit organizations that implement evidence- 
based programs and have a demonstrated record of helping 
schools to improve student outcomes. Organizations like the 
Boys and Girls Clubs, City Year, Communities In Schools, Big 
Brothers, Big Sisters, the United Way, Citizen Schools, College 



Summit, and growing numbers of after-school providers are 
highly effective at supporting the implementation of early 
warning systems. It would be beneficial to students if school 
districts and these effective organizations could utilize SES 
funds to support EWS. 

Use grant making to incentivize the integration of early 
warning indicator and intervention systems with related 
indicator and data driven improvement efforts. 

Federal and philanthropic grants should be aligned to reward 
effective best practices in data, particularly through Invest in 
Innovation, Race to theTop, and School Improvement Grants. 

This should include supporting the integration of EWS with 
student information systems and state longitudinal data systems. 

It is also important that EWS are integrated into efforts aimed 
at improving student readiness at entry to school and increasing 
post-secondary attainment. It is critical that grant-making efforts 
do not inadvertently create serial and/or unconnected parallel 
efforts at the state and district level as these entities pursue 
available funding opportunities, resulting in data systems and school 
improvement efforts that do not integrate easily with each other, 
even though they are pursuing related goals. For example, some 
states are closing or renaming dropout prevention offices, as 
college and career readiness offices. If this results in an integrated 
approach to keep students in school and prepare them to succeed 
in post-secondary schooling and training, it is an advantage. If it 
just switches focus and resources from one section of the pipeline 
to another, it is not. Finally, early warning indicator analysis should 
be paired with an examination of the characteristics of the most 
recent year’s dropouts, so data-driven recovery opportunities 
can be provided for the students who — even with the enhanced 
supports provided by EWS — dropout. 

Require states to collect information from school districts 
on chronic absenteeism and school districts with high 
levels of chronic absenteeism to develop district-wide 
plans to reduce it. 

In high poverty school districts the number and percent of 
students who miss a month or more of school can be staggering. 
Yet, it is often unknown because states and districts are not asked 
to report this data. Some states, like California, cannot require 
districts to report this information because it is not federally 
required and hence would be viewed as an unfunded mandate. The 
re-authorization of ESEA Title I, Part A, Section I I I I should be 
amended to ensure that chronic absenteeism is effectively tracked 
and those students with attendance rates less than 90 percent are 
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accurately identified as at risk of dropping out. Moreover, districts 
with high levels of chronic absenteeism should be required to 
develop district-level intervention plans. States which tie funding to 
attendance should make sure that schools that effectively decrease 
chronic absenteeism see an appropriate increase in funding. 



3) Accelerate Graduation Rates 
by Strengthening the Public 
Education System 

At the Local and State Level 

Keep data-based dropout prevention and recovery in the 
forefront of public attention, particularly because of the 
economic downturn. 

For minimal up front costs EWS can help nonprofits, schools, 
districts, and states better target their resources to students 
at risk of dropping out. EWS can minimize the costs of sweeping 
programming and help provide interventions to those most 
in need. 

At the State and Federal Level 

Update State and Federal accountability systems to 
leverage the highest-impact data. 

For minimal up front costs EWS can help nonprofits, schools, 
districts, and states better target their resources to students at 
risk of dropping out. EWS can minimize the costs of sweeping 
programming and help provide interventions to those most 
in need. 



At the Federal Level 

Update state and federal accountability systems to 
leverage the highest-impact data. 

This can be done initially through waivers, and permanently through 
the reauthorization of ESEA.Any states that receive relief from 
the current accountability systems should be held accountable 
for having a strong EWS.The reauthorization of ESEA, Title I, Part 
A should then ensure that states, districts, and schools have the 
resources they need to have strong EWS. 

• Further, the percent of students who earn on-time promotion 
from ninth to tenth grade (or fail one or fewer credit bearing 
courses) and the percent of students who are chronically 
absent should be added to emerging next generation state 
and federal accountability systems. As they emerge, the 
percentage of students meeting college- and career-ready 
benchmarks should also be added to these systems. To inform 
state and district improvement efforts and resource allocation, 
states should track the number and percent of students with 
off-track indicators and the number and percent of students 
being retained in grade at the school level. 

Clarify and better communicate the implications of 
Family Educational Rights and Privacy Act (FERPA) 
to stakeholders. 

The protection of children’s privacy must be balanced with 
the need for stakeholders’ support. When issuing new rules 
surrounding FERPA before the end of 201 I, the Department of 
Education should make clear that data systems can be tailored to 
ensure that parents, nonprofits, professional providers, and other 
state and local agencies only have access to the data they require 
to support the students they serve. By providing examples of 
steps states districts and schools can take to share the data that 
is appropriate to share, but protect the data that ought not to 
be shared, the Department of Education will help all education 
stakeholders understand the rules surrounding data, and help 
external organizations better support students. 
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Early Warning Indicator and Interventions Systems can become 
powerful tools in school systems and states across America to 
keep students on track to graduate from high school prepared 
for college and the workforce. Research shows we can identify 
which students, absent effective supports, have very low odds 
of graduating, early enough in their educational experience that 
we have sufficient time to intervene. It also shows that effective 
supports exist and that the challenge is organizing these supports 
in the most impactful and efficient means possible. This at its heart 
is what EWS are designed to do. With one in four of our nation’s 
public school students, and forty percent of minorities failing to 
graduate from high school, the development and implementation of 
EWS should become an urgent national priority. 



Although these systems are still in their early stages of 
development and testing, this period of rapid growth and 
innovation provides a laboratory for learning and gives schools, 
districts, and states the opportunity to accelerate their progress 
in providing meaningful data to educators. As a result, we can 
more effectively mobilize the supports students need to succeed 
in school and in life. The stakes are high for our children, school 
improvement efforts, states, and nation. In an increasingly 
competitive global workforce, we need all the tools available — 
including Early Warning Indicator and Intervention Systems — to 
keep students on track for success. 




Many individuals have been wonderfully helpful in sharing their 
experience with developing and implementing early warning 
indicator and intervention systems at multiple levels. We offer 
great thanks to them for their willingness to share lessons learned 
and next steps envisioned in their own organizations and for the 
country, including advisors in seven states:Alabama, California, 
Louisiana, Massachusetts, South Carolina, Texas, and Virginia and 
1 6 communities: Boston, Chattanooga, Chicago, Columbia, Dallas, 
Detroit, Houston, Knoxville, Memphis, Minneapolis, Nashville, New 
Orleans, Philadelphia, San Antonio, Seattle, and St. Louis. A special 
thanks also for the contributions of Carolyn Trager of City Year; 
Aimee Guidera and Lyndsay Pinkus of the Data Quality Campaign; 
Titus DosRemedios and Amy O’Leary of Early Education for All; 
Donna DiFillippo of Raising A Reader MA; Sujata Bhat of DC Prep; 
and the tireless Civic Enterprises team, including Megan Hoot, 
Frederic Brizzi, Aaron Gold.Tess Mason-Elder, Michael Poe, and 
Chris Wagner. 



Civic Enterprises is a public policy development firm dedicated to 
informing discussions of issues important to the nation. For more 
information, please visit www.civicenterprises.net 

The Everyone Graduate Center at Johns Hopkins University 
seeks to identify the barriers that stand in the way of all students 
graduating from high school prepared for adult success, to develop 
strategic solutions to overcome the barriers, and to build local 
capacity to implement and sustain them. For more information, 
please visit www.everylgraduates.org 
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APPENDIX I: 

THE CIVIC MARSHALL PLAN TO BUILD 
A GRAD NATION 

In the aftermath of World War II, Secretary of State George 
C. Marshall instructed George Kennan and his policy planning 
staff to “avoid trivia” in developing their plan to help rebuild 
Europe. A coalition of leading institutions has adopted this same 
approach in developing a “Civic Marshall Plan” to end the dropout 
epidemic and reach the national goal of having 90 percent of our 
students graduating from high school and obtaining at least one 



gradnat ion 
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year of postsecondary schooling or training by 2020. We believe 
that ending the dropout epidemic is possible because we now 
know which students are likely to drop out, absent effective 
interventions, and where these students go to school. We also 
know that evidence-based solutions exist.Thus, we are left with 
an engineering problem of getting the right supports to the right 
students in a timely fashion at the scale and intensity required. 

To meet this challenge, we need to take a targeted and phased 
approach, driven by our understanding of where the challenge is 
greatest and where concerted efforts can have the largest impact. 



Take Action within 
Low Graduation Rate 
Communities 



Build and Enable 
State and District 
Capacity to Improve 
Graduation and 
College Readiness 
Rates 



Accelerate 
Graduation Rates 
by Strengthening 
the Public Education 
System 



Start with Early Reading 


Focus on the Middle Grades 


Turn Around or Replace the Nation’s Dropout Factoris 


Harness the Power of Nonprofits to Provide Expanded Student Supports 


Link Researchers to Practitioners and Policy 


Build Early Warning and Intervention Systems 


Create a Multi-Sector and Community-Based Effort 


Enhance High School and College Graduation Rate Data 


Develop New Education Options based on Student and Community Needs and Interests 


Develop Parent and Family Engagement Strategies 


Elicit Perspectives of Students, Educators, and Parents 


Reauthorize the Elementary and Secondary Education Act 


Build Linked, Common Data Systems and Enhance Data-Driven Decision Making 


Set High Expectations and Provide Engaging Coursework 


Train and Support Highly Effective and Accountable Teachers 


Train and Support Highly Effective and Accountable Principals 


Connect the Post Secondary Completion Agenda with High School Graduation 
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Strategic, Tiered Approach 

Dropout Factory and Related Schools 

We will first target and help build the capacity of the states and 
school districts to transform or replace the remaining 1,634 
dropout factories and their feeder elementary and middle schools 
that account for half of the nation's dropouts, and the high schools 
with graduation rates between 61 and 75 percent. If the nation’s 
dropout factories and the high schools with graduation rates 
between 61 and 75 percent collectively increase their graduation 
rates by 20 percentage-points by 2020, amounting to an average of 
a two percentage-point increase per year, the nation will achieve 
its 90 percent graduation rate goal. 

Initial Benchmarks 

The Class of 2020 needs to earn 600,000 more high school 
diplomas than the Class of 2008 holding population growth 
constant). To ensure this happens, we will use a phased approach 
with clear goals. 

For 20 1 2: 

Substantially increase the number of struggling students reading at 
grade level by fifth grade; reduce chronic absenteeism; and conduct 
needs and capacity assessments of targeted schools. 

For 2013: 

Establish early warning and intervention systems in every targeted 
school district and state; re-design the middle grades as necessary 
to foster high student engagement and preparation for rigorous 
high school courses; and place a trained nonprofit school success 
mentor for every 1 5-20 students with off-track indicators. 



For 2016: 

Transform or replace the nation’s dropout factories and provide 
transition supports for struggling students in grades eight through 
10 in all schools with graduation rates below 75 percent; provide 
all students (including those who have dropped out) with clear 
pathways from high school to career training and college; and raise 
compulsory school age to 18 in all states. 

Next Steps 

Ending the dropout crisis will take a concerted effort by leaders 
and citizens at all levels of our communities, states, and nation. 
Progress over the last decade gives us confidence that good 
research can continue to guide our efforts, accurate data can 
prompt appropriate responses, and a targeted approach can help 
us reach our goals.The futures of millions of children are at stake, 
as are the health and vibrancy of our communities, economy, and 
nation. We should redouble our efforts to keep the high school 
dropout challenge a top national priority; mobilize the will, people, 
and resources to meet the challenge; and equip next generations 
with the knowledge and skills they need to find productive work 
and participate actively in American life. We have created a "Civic 
Marshall Plan Index” and will provide annual updates to keep track 
of our progress and challenges in ending the dropout crisis and 
building a grad nation. 



Americas Promise 
Alliance 
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i All IANCI FOR 
I XCIllENT I DUCAT ION 



56 



Appendices 



APPENDIX 2: 

ADDITIONAL RESOURCES ON 
DATA USAGE AND EARLY WARNING 
INDICATOR AND INTERVENTION 
SYSTEMS 

Resources with a Focus on EWS 

Alliance for Excellent Education, Civic Enterprises, and the 
Data Quality Campaign for AT&T, March 2011. Education as a 
Data-Driven Enterprise:A Primer for Leaders in Business, 
Philanthropy, and Education available at: 

www.civicenterprises.net/reports.php 

Balfanz, R. (20 10). Three Steps to Building an Early Warning 
and Intervention System for Potential Dropouts. PowerPoint 
presentation available at: 

www.everylgraduates.org/balfanz/item/95-three-steps- 

to-building-an-early-warning-and-intervention-system- 

for-potential-dropouts.html 

Balfanz, R. et al (2009). "Grad Nation:A Guidebook to Help 
Communities Tackle the Dropout Crisis.” Available a: 
www.civicenterprises.net/reports.php 

Balfanz, R. (2007). What Your Community Can Do to End Its 
Drop-Out Crisis: Learnings from Research and Practice. Johns 
Hopkins University. 

Developing a Dropout Early Warning and Intervention System 
(DEWIS) (no date). Office of Superintendant of Public Instruction, 
Washington Department of Education. 

www.kl2.wa.us/SecondaryEducation/BuildingBridges/ 
pubdocs/DE WISGuide-Final.pdf 

Early Warning System Implementation Guide (20 1 0). National 
High School Center.www.doe.virginia.gov/support/ 
school_improvement/early_warning_system/materials/ 
implementation_guide.pdf 



Heppen, J. B. & Therriault, S. B. (2008). Developing Early Warning 
Systems to Identify Potential High School Dropouts. National 
High School Center. 

Institute of Education Sciences (2008). IES Practice Guide: Dropout 
Prevention. Institute of Education Sciences National Center for 
Education Evaluation and Regional Assistance, U.S. Department 
of Education. 

Jerald, C. D. (2006). Identifying Potential Dropouts: Key Lessons 
for Building an Early Warning Data System. Achieve, Inc. & Jobs for 
the Future. 

Mac Iver, M.A. & Mac Iver, D. J. (2009). Beyond the Indicators: 

An Integrated School Level Approach to dropout prevention. 
Mid-Atlantic Equity Center. 

Princiotta, D. & Reyna, R. (2009). Achieving Graduation for All: 

A Governor’s Guide to Dropout Prevention and Recovery. 

National Governors Association Center for Best Practices. 

Using Student Achievement Data to Support Instructional Decision 
Making. (2009). Texas Consortium on School Research. 
http://ies.ed.gov/ncee/wwc/pdf/practiceguides/dddm_ 
Pg_092909.pdf 

Additional Resources on Longitudinal 
Data and Data Systems 

Data Quality Campaign (2009). A Policymaker’s Guide: Leveraging 
Longitudinal Student Data to Develop College and Career Ready 
High School Graduates. Data Quality Campaign. 

Data Quality Campaign (2009b). The Next Step: Using Longitudinal 
Data Systems to Improve Student Success. Data Quality Campaign. 

Data Quality Campaign (2010). Creating Reports Using 
Longitudinal data: How States Can Present Information to 
Support Student Learning and School System Improvement. 

Data Quality Campaign. 
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Dougherty, C. (2008). Information Won't Be Used If No One 
Can See It: Why States Should Publish Statistics based on 
Longitudinal Student Data. Data Quality Campaign. 

Laird, E. (2008). Developing and Supporting P-20 Education Data 
Systems: Different States, Different Models. Data Quality Campaign. 

Taveras, B„ Douwes, C. & Johnson, K. (20 1 0). New Visions for 
Public Schools: Using Data to Engage Families. Harvard Family 
Research Project. 



Resources on Privacy and 
FERPA Guidelines 

The Department of Education will be issuing new guidance on 
privacy, scheduled for the end of the 20 1 I calendar year. 

Though slightly out of date, the National Forum on Education 
Statistics has developed three publications related to privacy issues 
in education settings available at: 
http://nces.ed.gov/forum/ferpa_links.asp 

Additional information is available from the Thomas B. Fordham 
Institute including,“Getting FERPA Right: Encouraging Data Use 
While Protecting Student Privacy,” a chapter in the report: Byte 
at the Apple: Rethinking Education Data for the Post-NCLB Era, 
available at: www.edexcellence.net/publications-issues/ 
publications/a-byte-at-the-apple.html 
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